Posts tagged “rhel”.

Installing and booting VMware ESXi over FCoE on an IBM BladeCenter H with Brocade CNA

Hardware:
– IBM BladeCenter H
– HS22V and HX5 blades with Brocade 2 port 10GbE Converged Network Adapter, no local drives
– IBM Storwize V7000

At the time of writing the Brocade 2 port 10GbE Converged Network Adapter for IBM BladeCenter (81Y1650) is supported on VMware ESXi up to version 4.1.

To install VMware on the blades, a custom VMware ESXi 4.1 ISO image with Brocade CNA drivers is required.

More… »

Exactly what I wanted to see on a production server…

RHEL 6.1 / Scientific Linux 6.1 dmesg:

TECH PREVIEW: fscache may not be fully supported.
Please review provided documentation for limitations.
FS-Cache: Loaded
FS-Cache: Netfs 'nfs' registered for caching

Excerpt from RHEL 6.1 Technical Notes, Chapter 3. Technology Previews:

FS-Cache
FS-Cache is a new feature in Red Hat Enterprise Linux 6 that enables networked file systems (e.g. NFS) to have a persistent cache of data on the client machine.

… and nothing about the limitations or how to disable fscache.

Tried to add install fscache /bin/true to /etc/modprobe.d/fscache.conf with the following dmesg result (of course, no NFS filesystems mounted):

nfs: Unknown symbol __fscache_read_or_alloc_pages
nfs: Unknown symbol __fscache_relinquish_cookie
nfs: Unknown symbol __fscache_unregister_netfs
nfs: Unknown symbol __fscache_maybe_release_page
nfs: Unknown symbol __fscache_read_or_alloc_page
nfs: Unknown symbol __fscache_uncache_page
nfs: Unknown symbol __fscache_register_netfs
nfs: Unknown symbol __fscache_write_page
nfs: Unknown symbol __fscache_wait_on_page_write
nfs: Unknown symbol __fscache_acquire_cookie
nfs: Unknown symbol __fscache_read_or_alloc_pages
nfs: Unknown symbol __fscache_relinquish_cookie
nfs: Unknown symbol __fscache_unregister_netfs
nfs: Unknown symbol __fscache_maybe_release_page
nfs: Unknown symbol __fscache_read_or_alloc_page
nfs: Unknown symbol __fscache_uncache_page
nfs: Unknown symbol __fscache_register_netfs
nfs: Unknown symbol __fscache_write_page
nfs: Unknown symbol __fscache_wait_on_page_write
nfs: Unknown symbol __fscache_acquire_cookie

Solved: e1000: eth0: e1000_clean_tx_irq: Detected Tx Unit Hang

On one amd64 machine running CentOS 5.5 x86_64 the e1000 network interfaces go down and up with the following messages:

e1000: eth0: e1000_clean_tx_irq: Detected Tx Unit Hang
 Tx Queue             <0>
 TDH                  <62>
 TDT                  <8d>
 next_to_use          <8d>
 next_to_clean        <62>
buffer_info[next_to_clean]
 time_stamp           <10037f7b6>
 next_to_watch        <62>
 jiffies              <10037fcd4>
 next_to_watch.status <0>
NETDEV WATCHDOG: eth0: transmit timed out

This issue usually appears on machines with 4GB or more memory. I have tried a lot of things but no luck.

The fix comes with new Intel’s e1000 driver.

Download the latest driver from e1000 stable on SF (8.0.25 at the time of writing).

Build the RPM:

rpmbuild -tb /path/to/e1000-8.0.25.tar.gz

Install the new e1000 RPM driver:

rpm -ivh /usr/src/redhat/RPMS/x86_64/e1000-8.0.25-1.x86_64.rpm

Add ignore_64bit_dma=1 driver option in /etc/modprobe.conf:

options e1000 ignore_64bit_dma=1

Reboot and enjoy!

From e1000 README file:

ignore_64bit_dma
—————-
Valid Range:   0-xxxxxxx (0=off)
Default Value: 0
Usage: insmod e1000.ko ignore_64bit_dma=1

When non zero the driver will only request DMA mapping of host memory
in the lower 4GB region. This provides a workaround for users of AMD platforms
GA-MA78G-DS3H & SM4021M-T2R+ that have reported TXHangs on system that have
>4GB RAM, suspected caused by some (no deep root cause) issue in the Dual
Address Cycle (DAC) DMA mechanism needed to access addresses above 4GB.
Setting ignore_64bit_dma to 1 activates the workaround.

This parameter is different than other parameters, in that it is a
single (not 1,1,1 etc.) parameter applied to all driver instances and
it is also available during runtime at
/sys/module/e1000/parameters/ignore_64bit_dma

Update: if your newer kernel crashes with e1000 ElRepo driver version 8.0.30 or older (detailed here), please update to the latest version, currently 8.0.35. Thanks again ElRepo!