I would (just like you I suppose) be very much in favor of a Hobbit addaptation that is true to the book and doesn't make major changes about it. Glimpses at characters from the lord of the rings who were probably present on the stages of the Hobbit without playing a major role (Legolas, Arwen, Gimli) would not hurt, if this was all it took to make The Hobbit a "Lord of the Rings 4" (or should that be a Lord of the Rings 1/2?). There is much in The Hobbit which I expect could be presented very impressively. The lake town, Smaug, Beorn, and the battle of the five armies for example; not to mention the treasures Smaug guarded
There are some places and events only vaguely mentioned in the Hobbit and the Lord of the Rings. For some reason I've grown very interested in Dol Guldur, Sauron's fortress in Southern Mirkwood. Perhaps my interest in that stronghold is caused by the very lack of information on it. Nevertheless I suppose that any reference beyond a possible flashback (about how Gandalf was given the map of Erebor) would lead too far from the story.
PS: The Hobbit flashback from the Lord of the Rings deviated from the books. We saw Bilbo find the Ring and hear Gollum noticing its loss almost the same moment. That however was probably just to cut a long story short in the lord of the rings. They wouldn't dare to make such a crucial change in an actual Hobbit movie I hope.