Every failing sample below is the real model output, with the facts it missed or invented. Trial 0 runs at temperature 0, trials 1 and 2 at 0.7 — so a task passing 1/3 is a sampling-luck pass, not a reliable one. Models are ordered best first. Other technologies: C · Python · Bash · research · how to reproduce.
Setup and model serving are on the models index — including a script-free version using nothing but curl and a compiler; with a server running, this is the exact command that produced the samples below:
# research paper: extract NFS facts from ~2,100 words of RHEL 10 docs
# (distractor paragraphs included), 3 trials, graded by exact fact match
python scripts/eval_research.py --target <model> --trials 3 \
--dump-failures results/failuresServed via llama.cpp CUDA · 33.6 tok/s · 36.3 GB RAM · coding total 111/126
No failures.
Served via llama.cpp CUDA · 51.2 tok/s · 8.9 GB RAM · coding total 108/126
No failures.
Served via llama.cpp CUDA · 35.3 tok/s · 4.9 GB RAM · coding total 108/126
- **NFS with TLS support added (RHEL 10.0)** – TLS is now fully supported for NFS, encrypting RPC traffic between clients and servers. This improves security without requiring application changes, though it depends on kernel TLS (kTLS), which is still a Technology Preview.
- **NFS client now supports `vers` as an alias for `nfsvers` (RHEL 10.0)** – The `vers` mount option is now equivalent to `nfsvers`, improving compatibility with existing scripts and configurations that use the shorter option name.
- **Increased default and maximum `rsize`/`wsize` values for NFS (RHEL 10.0)** – The maximum transfer size for NFS read/write operations is now 1,048,576 bytes on both client and server, improving performance by allowing larger data transfers per operation.
- **IPv6 address support fixed in NFS URLs for ReaR (RHEL 10.0)** – Square brackets around IPv6 addresses in `nfs://` URLs are now correctly parsed, enabling backup and restore operations to NFS servers over IPv6 without workarounds.
- **FS-Cache integration with NFS remains supported (RHEL 10.0)** – Client-side caching of NFS content via FS-Cache and `cachefilesd` is documented and supported, allowing local caching of remote NFS data to reduce network and server load, with support for XFS, ext3, and ext4 as cache back ends.- **NFS with TLS support added (RHEL 10.0)**: NFS now supports TLS encryption for RPC traffic, enhancing security. Relies on kernel TLS (kTLS), which is a Technology Preview.
- **NFS client supports larger rsize/wsize (RHEL 10.0)**: Maximum read/write buffer size increased to 1,048,576 bytes, improving NFS client performance.
- **Fixed: IPv6 address parsing in nfs:// URLs (RHEL 10.0)**: ReaR now correctly handles IPv6 addresses in `nfs://` URLs without requiring workarounds, fixing premature abort errors.
- **FS-Cache client-side caching documented (RHEL 10.0)**: NFS performance can be improved using FS-Cache with `cachefilesd`, reducing server load and network usage by caching data locally.- **NFS with TLS support added (RHEL 10.0)**: NFS now supports TLS encryption for RPC traffic, enhancing security. Relies on kernel TLS (kTLS), which is a Technology Preview.
- **NFS client supports larger rsize/wsize (RHEL 10.0)**: Maximum read/write buffer size increased to 1,048,576 bytes, improving NFS client performance.
- **Fixed: IPv6 address parsing in nfs:// URLs (RHEL 10.0)**: ReaR now correctly handles IPv6 addresses in `nfs://` URLs without requiring workarounds, fixing premature abort errors.
- **FS-Cache client-side caching documented (RHEL 10.0)**: NFS performance can be improved using FS-Cache with `cachefilesd`, reducing server load and network usage by caching data locally.Served via llama.cpp CUDA · 54.2 tok/s · 3.8 GB RAM · coding total 107/126
No failures.
Served via llama.cpp CUDA · 16.4 tok/s · 9.2 GB RAM · coding total 105/126
- **NFS with TLS support is fully supported**: NFS now supports Transport Layer Security (TLS) for encrypting RPC traffic between clients and servers, enhancing security for NFS communications. This matters because it protects data in transit from eavesdropping and tampering, meeting modern security requirements for sensitive workloads.
*Note: This relies on kernel TLS (kTLS), which is a Technology Preview in RHEL 10.*
- **IPv6 address handling in NFS URLs fixed**: Previously, ReaR (Relax-and-Recover) failed to parse IPv6 addresses in `nfs://` URLs when enclosed in square brackets due to incorrect interpretation of brackets as shell metacharacters. This fix allows proper use of IPv6 addresses in NFS URLs (e.g., `nfs://[2001:db8:ca2:6::101]/path`) without workarounds, enabling reliable backup and recovery operations in IPv6-only or dual-stack environments.
- **NFS client-side caching via FS-Cache is documented and enabled**: The guide details how FS-Cache, backed by `cachefilesd`, enables transparent client-side caching of NFS content to improve performance, reduce server load, and allow efficient local data retrieval. It works with XFS, ext3, and ext4 as cache back ends and can cache partial files while hiding cache I/O errors from the client. This matters because it significantly reduces network traffic and server load for repetitive read workloads, improving scalability and responsiveness—especially beneficial in virtualized or high-latency network environments.
*Note: While enabling caching may slightly slow the client due to cache lookup overhead, the net gain in reduced network/server load typically outweighs this cost.*- **NFS with TLS support is fully supported**: NFS now supports Transport Layer Security (TLS) for encrypting RPC traffic between clients and servers, enhancing security for NFS communications. This matters because it protects data in transit from eavesdropping and tampering, meeting modern security requirements for sensitive workloads.
*Note: This relies on kernel TLS (kTLS), which is a Technology Preview in RHEL 10.*
- **IPv6 address handling in NFS URLs fixed**: Previously, ReaR (Relax-and-Recover) failed to parse IPv6 addresses in `nfs://` URLs when enclosed in square brackets due to incorrect interpretation of brackets as shell metacharacters. This fix allows proper use of IPv6 addresses in NFS URLs (e.g., `nfs://[2001:db8:ca2:6::101]/path`) without workarounds, enabling reliable backup and recovery operations in IPv6-only or dual-stack environments.
- **NFS client-side caching via FS-Cache is documented and enabled**: The guide details how FS-Cache, backed by `cachefilesd`, enables transparent client-side caching of NFS content to improve performance, reduce server load, and allow efficient local data retrieval. It works with XFS, ext3, and ext4 as cache back ends and can cache partial files while hiding cache I/O errors from the client. This matters because it significantly reduces network traffic and server load for repetitive read workloads, improving scalability and responsiveness—especially beneficial in virtualized or high-latency network environments.
*Note: While enabling caching may slightly slow the client due to cache lookup overhead, the net gain in reduced network/server load typically outweighs this cost.*- **NFS with TLS support is fully supported**: NFS now supports Transport Layer Security (TLS) for encrypting RPC traffic between clients and servers, enhancing security for NFS communications. This matters because it protects data in transit from eavesdropping and tampering, meeting modern security requirements for sensitive workloads.
*Note: This relies on kernel TLS (kTLS), which is a Technology Preview in RHEL 10.*
- **IPv6 address handling in NFS URLs fixed**: Previously, ReaR (Relax-and-Recover) failed to parse IPv6 addresses in `nfs://` URLs when enclosed in square brackets due to incorrect interpretation of brackets as shell metacharacters. This fix allows proper use of IPv6 addresses in NFS URLs (e.g., `nfs://[2001:db8:ca2:6::101]/path`) without workarounds, enabling reliable backup and recovery operations in IPv6-only or dual-stack environments.
- **NFS client-side caching via FS-Cache is documented and enabled**: The guide details how FS-Cache, backed by `cachefilesd`, enables transparent client-side caching of NFS content to improve performance, reduce server load, and allow efficient local data retrieval. It works with XFS, ext3, and ext4 as cache back ends and can cache partial files while hiding cache I/O errors from the client. This matters because it significantly reduces network traffic and server load for repetitive read workloads, improving scalability and responsiveness—especially beneficial in virtualized or high-latency network environments.
*Note: While enabling caching may slightly slow the client due to cache lookup overhead, the net gain in reduced network/server load typically outweighs this cost.*Served via llama.cpp CUDA · 81.0 tok/s · 3.7 GB RAM · coding total 102/126
- **NFS with TLS support (new feature)** – RHEL 10 now fully supports TLS for NFS RPC traffic via kernel TLS (kTLS). This encrypts all client‑server communication, improving security and compliance, while adding a modest encryption overhead that can affect throughput on high‑latency links.
- **IPv6 address parsing bug fix for `nfs://` URLs** – Previously, ReaR treated square brackets around IPv6 addresses as shell metacharacters, causing `nfs://[2001:db8::1]/…` URLs to fail and abort the backup/restore process. The fix now correctly parses bracketed IPv6 addresses, enabling reliable use of NFS mounts over IPv6 in backup scripts and other tooling.
- **FS‑Cache client‑side caching for NFS (new feature)** – RHEL 10 introduces FS‑Cache support for NFS via the `cachefiles` back‑end and `cachefilesd` daemon. When enabled, NFS data is cached locally on an ext3/4/XFS filesystem, reducing network traffic and server load and improving read performance for frequently accessed files. The trade‑off is that cache maintenance can add overhead and may slow the client if the cache is not properly sized or if many large files bypass the cache.- **NFS with TLS support (new feature)** – RHEL 10 now fully supports TLS for NFS RPC traffic, providing encrypted client‑server communication. This improves security and can affect performance due to the encryption overhead, but it enables secure NFS deployments that were previously limited to plaintext.
- **IPv6 address parsing bug fix for nfs:// URLs** – The kernel now correctly interprets IPv6 addresses enclosed in square brackets in `sshfs://` or `nfs://` URLs used in `BACKUP_URL`/`OUTPUT_URL`. This resolves a parsing error that caused ReaR to abort, allowing administrators to use IPv6 NFS endpoints without workarounds.
- **FS‑Cache client‑side caching for NFS (new feature)** – RHEL 10 introduces FS‑Cache support for NFS mounts, enabling local caching of NFS data via the `cachefiles` backend. This reduces network traffic, lowers server load, and can significantly improve read performance for frequently accessed files.- **NFS with TLS support (new feature)** – RHEL 10 now fully supports TLS for NFS RPC traffic, providing encrypted client‑server communication. This improves security and can affect performance due to the encryption overhead, but it enables secure NFS deployments that were previously limited to plaintext.
- **IPv6 address parsing bug fix for nfs:// URLs** – The kernel now correctly interprets IPv6 addresses enclosed in square brackets in `sshfs://` or `nfs://` URLs used in `BACKUP_URL`/`OUTPUT_URL`. This resolves a parsing error that caused ReaR to abort, allowing administrators to use IPv6 NFS endpoints without workarounds.
- **FS‑Cache client‑side caching for NFS (new feature)** – RHEL 10 introduces FS‑Cache support for NFS mounts, enabling local caching of NFS data via the `cachefiles` backend. This reduces network traffic, lowers server load, and can significantly improve read performance for frequently accessed files.Served via llama.cpp CUDA · 55.8 tok/s · 6.6 GB RAM · coding total 102/126
No failures.
Served via llama.cpp CUDA · 11.0 tok/s · 9.6 GB RAM · coding total 102/126
No failures.
Served via llama.cpp CUDA · 59.6 tok/s · 9.3 GB RAM · coding total 102/126
No failures.
Served via llama.cpp CUDA · 93.6 tok/s · 4.3 GB RAM · coding total 102/126
No failures.
Served via llama.cpp CUDA · 11.2 tok/s · 9.3 GB RAM · coding total 99/126
No failures.
Served via llama.cpp CUDA · 14.3 tok/s · 1.4 GB RAM · coding total 98/126
No failures.
Served via llama.cpp CUDA · 68.8 tok/s · 9.3 GB RAM · coding total 96/126
No failures.
Served via llama.cpp CUDA · 67.4 tok/s · 1.7 GB RAM · coding total 96/126
* **NFS with TLS support added:** NFS traffic can now be encrypted using Transport Layer Security (TLS) via kernel TLS (kTLS). This matters because it enhances security by ensuring encrypted communication between clients and servers without requiring application-level changes.
* **IPv6 address support in NFS URLs fixed:** The ReaR tool now correctly interprets IPv6 addresses (enclosed in square brackets) in `nfs://` URLs. This matters because it resolves a bug that caused ReaR to abort when using IPv6 addresses in backup or output URLs.
* **NFS rsize and wsize maximums increased:** The maximum transfer size for NFS read and write operations is now 1,048,576 bytes (1 MiB). This matters because it allows larger data chunks to be transferred in a single operation, potentially improving throughput and reducing the number of round trips.
* **NFS client-side caching (FS-Cache) available:** The `cachefilesd` service can be used to cache NFS content on the client side. This matters because it reduces network and server load by satisfying read requests locally and enables efficient local data retrieval.
* **NFS mount options documentation updated:** The `vers` option is now documented as an alias for `nfsvers` for compatibility. This matters because it clarifies that both options function identically for NFS version selection.Served via llama.cpp CUDA · 12.6 tok/s · 9.3 GB RAM · coding total 96/126
No failures.
Served via llama.cpp CUDA · 22.4 tok/s · 9.4 GB RAM · coding total 93/126
No failures.
Served via llama.cpp CUDA · 91.1 tok/s · 2.1 GB RAM · coding total 90/126
No failures.
Served via llama.cpp CUDA · 50.8 tok/s · 3.4 GB RAM · coding total 88/126
No failures.
Served via llama.cpp CUDA · 43.1 tok/s · 1.3 GB RAM · coding total 83/126
No failures.
Served via llama.cpp CUDA · 9.1 tok/s · 8.8 GB RAM · coding total 81/126
- **Fixed parsing of IPv6 addresses in NFS URLs**: ReaR now correctly interprets IPv6 addresses enclosed in square brackets (e.g., `nfs://[2001:db8- **Fixed parsing of IPv6 addresses in NFS URLs**: ReaR now correctly interprets IPv6 addresses enclosed in square brackets (e.g., `nfs://[2001:db8Served via llama.cpp CUDA · 8.8 tok/s · 9.7 GB RAM · coding total 73/126
- **NFS with TLS support**: RHEL 10 offers NFS with Transport Layer Security (TLS), enhancing security by enabling encrypted communication between clients and servers. This feature relies on kernel TLS (kTLS), provided as a Technology Preview.
- **IPv6 address parsing fix**: A bug affecting the use of IPv6 addresses in sshfs:// and nfs:// URLs has been fixed. Now, square brackets around IPv6 addresses are correctly interpreted, allowing their use in these URL schemes.
- **NFS client-side caching**: RHEL 10 introduces client-side caching for NFS using FS-Cache, which can improve performance, reduce server load, and enable efficient local data retrieval. FS-Cache requires a compatible cache back-end file system (XFS, ext3, or ext4) and can be configured using the`/etc/cachefilesd.conf` file.<|END_OF_TURN_TOKEN|>- **NFS with TLS support**: RHEL 10 offers NFS with Transport Layer Security (TLS), enhancing security by enabling encrypted communication between clients and servers. This feature relies on kernel TLS (kTLS), provided as a Technology Preview.
- **IPv6 address parsing fix**: A bug affecting the use of IPv6 addresses in sshfs:// and nfs:// URLs has been fixed. Now, square brackets around IPv6 addresses are correctly interpreted, allowing their use in these URL schemes.
- **NFS client-side caching**: RHEL 10 introduces client-side caching for NFS using FS-Cache, which can improve performance, reduce server load, and enable efficient local data retrieval. FS-Cache requires a compatible cache back-end file system (XFS, ext3, or ext4) and can be configured using the`/etc/cachefilesd.conf` file.<|END_OF_TURN_TOKEN|>- **NFS with TLS support**: RHEL 10 offers NFS with Transport Layer Security (TLS), enhancing security by enabling encrypted communication between clients and servers. This feature relies on kernel TLS (kTLS), provided as a Technology Preview.
- **IPv6 address parsing fix**: A bug affecting the use of IPv6 addresses in sshfs:// and nfs:// URLs has been fixed. Now, square brackets around IPv6 addresses are correctly interpreted, allowing their use in these URL schemes.
- **NFS client-side caching**: RHEL 10 introduces client-side caching for NFS using FS-Cache, which can improve performance, reduce server load, and enable efficient local data retrieval. FS-Cache requires a compatible cache back-end file system (XFS, ext3, or ext4) and can be configured using the`/etc/cachefilesd.conf` file.<|END_OF_TURN_TOKEN|>