Stealing freely from a link on curious frog (maybe I should call it riffing... sounds less dishonorable) I was inspired to use MapQuest to seek out this image of the area around the White House. Looks like DC planners were fans of the pentagram. (That link is the aerial photo. This one is the road map)
No need to jump to satanic conclusions.
Although the Roman Church has ascribed the goat as an image of their Devil, it has long been associated with fertility and fecundity, two qualities which were highly revered by ancient people.The pentagram was also a masonic symbol.
Some people take this information and run with it. Knock yourself out, pally.
Websurfing can be fun! I found out that the inverted pentagram inscribed with the goat head is called the "Sigil of Baphomet." Baphomet from the Greek "baphe" and "mentis" meaning "absorption of knowledge." Used by inquisitors trying to pry information out of their victims.
Interesting blog connection: Steve has a post about Christian iconography over at Absit Invidia. When I was a kid, I remember going on a retreat with my church and we met a fellow who was a researcher into iconography. Before talking to him, I thought an icon was simply an image that was somewhat visually representative of the object or action it stood in for -- the same idea icons are used for in computers. I has no idea what a religious icon was. His presentation was among the most interesting experiences I had in the church.