Based on Olympic games held in ancient Greece centuries ago (starting in 776 B.C.), the "Modern" Olympic era began with the games of 1896, held, naturally, in Athens, Greece.
After the initial success, the Olympics struggled for a few years, and it wasn't until 1904, that the Games again attracted a broad international field of participants--in 1904, 80% had been American--and great public interest, thereby marking the beginning of a rise in popularity and size of the Games.
From the 241 participants from 14 nations in 1896, the Games grew to nearly 11,100 competitors from 202 countries at the 2004 Summer Olympics in Athens.
The first major Summer Olympics (there were no Winter Games until 1924) were held in London in 1908. The second, held in Stockholm, Sweden in 1912, produced one of the first "Olympic controversies" when the great (Native) American athlete Jim Thorpe was ordered to surrender the Olympic medals he had won, when it was discovered that he had briefly played professional baseball (Professional athletes were, of course, not allowed in the Olympics then). The next few Games, including the first held in the U.S., the 1932 Los Angeles Games, were free of controversy, as far as I know. In 1936, on the eve of WWII, Berlin staged the Games, where another great U.S. athlete, Jesse Owens, was able to show up Adolf Hitler's Nazi beliefs by performing extraordinarily on the track and in the field. There were no Games during the ensuing war, but when they resumed in London in 1948, and through the 1952, 1956, 1960, and 1964 Games, things seemed pretty calm off the track. Most of the tension came from the increasing rivalry between the U.S. and USSR teams.
Then came the 1968 Games in Mexico City and things have really never been the same. Among other things, the '68 Games are remembered for the "Black Power" salute that Americans John Carlos and Tommie Smith executed on the Medal podium during the Star Spangled Banner. The Munich Olympics in 1972 were just one bad situation after another for the U.S. team, but that all paled significantly compared to the murder of several members of the Israeli team - one of the first times we became aware of the word "terrorist," though certainly not the last. I don't remember much negative about the 1976 Games in Montreal, but the U.S. boycotted the 1980 Games in Munich because of USSR policies, and the Soviet Union reciprocated by boycotting the 1984 Los Angeles Games.
To tell you the truth, I just haven't followed the Olympics all that closely since then. When I was a kid, I loved watching the Olympics and the National political conventions every four years. Neither are the same these days. The Atlanta Games of 1996 were marked by a bombing, and the Sydney Games of 2000, after the fact, by steroid usage and medal-stripping, a la Jim Thorpe.
This summer, the Games will be held, for the first time ever, in China. Already we have had a lot of protests, mainly because of Chinese political policy and abuses of human rights in Tibet and Darfur. The Prime Minister of Great Britain is not planning on attending Opening Ceremonies, there have been physical protests in London and Paris when the Olympic Torch- that symbol of good will - came to town, and in San Francisco, they had to re-route the Torch Run to avoid a similar situation there.
I no longer have any idea what or where "the Olympic Spirit" is.