Bug 588027 removed <deck id="contentDeck">. That made impossible to replace the right panel in places.xul (at least without ugly workarounds).
As an example, this is the overlay that I'm using in my extension (Firefox 3.6-6.0, broken with Firefox 7):
Looking at the new DOM structure, the deck is no longer covering all of the area to the right, only the panel at the bottom right. The new deck id is "detailsDeck".
Does your overlay require to replace the Library tree or just the panel on the bottom right? If it's the latter, then you just need to make a few adjustments. If not, then I agree that you need some ugly tricks to make it work, although I'm not sure how it makes sense to replace the whole Library tree given that it is the primary focus of that window.
Created attachment 551905 [details]
My extensions adds an item to the left pane and an entirely new right panel.