Mice and trackballs used to be simple; now, thanks to Microsoft, they're complicated. Fortunately, XFree86 is now pretty good at autodetecting your mouse when it starts up, so if that's what you're using you can skip the next section.
In the beginning, there was only the Mouse Systems 3-button serial mouse; this reported status to a serial port 30 times a second using a 5-byte serial packet encoding now called ``C'' protocol. The Logitech Series 7 and 9 mice were Mouse Systems-compatible. All Unixes that have any mouse support at all understand C-protocol serial mice.
Then Microsoft got into the act. They designed a two-button serial mouse which reports only deltas in a three-byte packet; that is, it sends changes in button status and motion reports only when the mouse is actually moving. This is called `M' protocol. Microsoft sold a lot of mice, so Logitech switched from `C' to `M' — but they added a third button, state changes for which show up in an optional fourth byte. Thus, `M+' protocol, upward-compatible with Microsoft's `M'. Most Unix vendors add support for M+ mice, but it's wise to check (Linux's XFree86 has it).
Bus mice are divided into 8255 and InPort types. These report info continuously at 30 or 60 Hz (though InPort mice have an option for reporting deltas only), and you get interrupts on events and then have to poll hardware ports for details. Bus mice are no longer widely available and there is no good reason to bother with them.
In addition to serial mice and bus mice, there are ``keyboard mice''. On PS/2s there are two identical-looking keyboard ports, labeled (with icons) ``mouse'' & ``keyboard''. Both are 6-pin mini-DINs that look like the regular PC keyboard port only smaller. Physically, the connector goes to the keyboard processor (often an 8042); electrically, it provides TTL levels (serial clock, serial data, ground and +5V); logically, it uses the same protocol as normal serial mice. The same keyboard processor that decodes the keyboard decodes the mouse. PS/2s have this port, many newer motherboards do as well. Confusingly, XFree86 thinks these keyboard mice are bus mice.
All things considered, Unix users are probably best off going with a serial mouse if their motherboard isn't one of the newer ones with a built-in PS/2 mouse. Most current clone motherboards give you two serial ports, so you can still dedicate one to this and still have one for the all-important modem. Not only are the compatibility issues less daunting, but a serial mouse loads the multitasking system less due to interrupt frequency.
Beware that most clone vendors, being DOS oriented, bundle two-button M-type mice. Thus, you may have to buy your own three-button mouse. Ignore the adspeak about dpi and pick a mouse or trackball that feels good to your hand.
Your humble editor really, really likes the Logitech TrackMarble, an optical trackball that eliminates the chronic roller-fouling problems of the older TrackMan. They're well-supported by XFree86 (type MouseMan), so any Linux or BSD will accept them.