Something I learned today: if you want to pick a 3D object from mouse coordinates, you can go from screen space coordinates to object space but you need to pay attention to the normalization of the screen space coordinates.
First, to get the screen space coordinate, you can use the mouse location x,y normalized on ( -1..1, -1..1 ) range. You need to supplement this with a z-coordinate that you get from the depth buffer.
glReadPixels( scrx, scry, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &depth);
Where I initially went wrong is, not realizing that this z coordinate needs to be remapped as well, from a 0..1 to a -1..1 range. The hint that lead me to this was looking at this code.
All you then need to do is multiply a matrix with this screen space coordinate to get a model space coordinate. The matrix to multiply is of course the inverse of the model-view-projection matrix. Simple enough to do yourself, so you do not need to link to libGLU just for its gluUnproject() function.
Oh, and do not forget to divide the x,y,z of the multiplication result by its w component.