fxp(4) - please test

["12" <bogdan_(_at_)_eastonline_(_dot_)_ru>]

hi, all

OpenBSD's fxp(4) driver don't yet enable use of PCI Memory Write and Invalidate
command on controllers for storing incoming packets into the host memory.
Compared to Memory Write command, Memory Write and Invalidete uses optimisied
writes to memory with a minimum transfer of one cache line (typically 8 or 16 
DWORDs for fxp(4) controllers). In other words, packet can be transferred by 
the bus master into the memory by one (when in best condition) command, thus 
reducing PCI bandwidth usage. It also allows memory performance optimization by
invalidating actual write-back cycle in cache contoroller thus shortening access
time.

This diff can be useful for an OpenBSD systems with a lot (5-6) of fxp NIC's
(sitting on one system PCI 32bit/33MHz bus), that acts as a heavy loaded 2'nd
layer bridge or a router/firewall (or any other system, where the PCI bus
itself can be a perfromance bottleneck).

Please test this diff on any combination of PCI fxp contollers and PCI bridges
you have and report any problems (or success stories) to me or to this list.

To test, use your favorite network bandwidth measurment tool and watch for
network latency and total throughput when in peak load..


Index: dev/pci/if_fxp_pci.c
===================================================================
B
RCS file: /home/cvs/OpenBSD/src/sys/dev/pci/if_fxp_pci.c,v
retrieving revision 1.31
diff -u -r1.31 if_fxp_pci.c
--- dev/pci/if_fxp_pci.c	20 Sep 2004 04:24:00 -0000	1.31
+++ dev/pci/if_fxp_pci.c	25 Dec 2004 16:26:29 -0000
@@ -237,6 +237,20 @@
 	    PCI_COMMAND_MASTER_ENABLE |
 	    pci_conf_read(pa->pa_pc, pa->pa_tag, PCI_COMMAND_STATUS_REG));
 
+	/*
+	 * enable PCI Memory Write and Invalidate command
+	 */
+	if (rev >= FXP_REV_82558_A4)
+		if (PCI_CACHELINE(pci_conf_read(pa->pa_pc, pa->pa_tag,
+		    PCI_BHLC_REG))) {
+			pci_conf_write(pa->pa_pc, pa->pa_tag,
+			    PCI_COMMAND_STATUS_REG,
+			    PCI_COMMAND_INVALIDATE_ENABLE |
+			    pci_conf_read(pa->pa_pc, pa->pa_tag,
+			    PCI_COMMAND_STATUS_REG));
+			sc->sc_flags |= FXPF_MWI_ENABLE;
+		}
+
 	/* Do generic parts of attach. */
 	if (fxp_attach_common(sc, intrstr)) {
 		/* Failed! */
Index: dev/ic/fxp.c
===================================================================
RCS file: /home/cvs/OpenBSD/src/sys/dev/ic/fxp.c,v
retrieving revision 1.63
diff -u -r1.63 fxp.c
--- dev/ic/fxp.c	23 Dec 2004 19:40:21 -0000	1.63
+++ dev/ic/fxp.c	25 Dec 2004 16:19:37 -0000
@@ -1312,6 +1312,9 @@
 		cbp->stripping |= 0x01;		/* truncate rx packets */
 	}
 
+	if (sc->sc_flags & FXPF_MWI_ENABLE)
+		cbp->void1 |= 0x01;		/* enable PCI MWI command */
+
 	if(!sc->phy_10Mbps_only)			/* interface mode */
 		cbp->mediatype |= 0x01;
 	else
Index: dev/ic/fxpvar.h
===================================================================
RCS file: /home/cvs/OpenBSD/src/sys/dev/ic/fxpvar.h,v
retrieving revision 1.20
diff -u -r1.20 fxpvar.h
--- dev/ic/fxpvar.h	23 Dec 2004 19:40:21 -0000	1.20
+++ dev/ic/fxpvar.h	26 Dec 2004 07:43:29 -0000
@@ -104,6 +104,7 @@
 	struct mbuf *rfa_headm;		/* first mbuf in receive frame area */
 	struct mbuf *rfa_tailm;		/* last mbuf in receive frame area */
 	int sc_flags;			/* misc. flags */
+#define	FXPF_MWI_ENABLE		0x10	/* enable use of PCI MWI command */
 #define	FXPF_DISABLE_STANDBY	0x20	/* currently need to work-around */
 #define	FXPF_UCODE		0x40	/* ucode load already attempted */
 	struct timeout stats_update_to; /* Pointer to timeout structure */