22 billion images. 100 countries. 10 million miles.
One day in 2003, Larry Page bought a consumer video camera, strapped it to his car window, and drove around San Francisco filming. He showed the footage to fellow engineers. “I bet we can do something with this.”
Nineteen years later, Google Street View had photographed the equivalent of 42 trips around the Earth, accidentally intercepted 600 GB of Wi-Fi traffic along the way, been told by the Swiss Federal Supreme Court that “99% isn’t enough,” effectively retreated from Germany, and re-photographed all of Japan.
This project has no ending. Roads around the world keep growing every day.
Mission
In the early 2000s, digital maps were still just “maps.” MapQuest and Google Maps alike were made of lines, dots, and labels.
In 2001, the “City Block Project” began at Stanford University’s Computer Graphics Laboratory. It was a research project sponsored by Google. Led by Marc Levoy, a professor known for rendering theory, the question was simple.
If you could see every road on Earth through photographs, how would maps change?
In 2003, Larry Page started acting on his own idea. He strapped a video camera to a car, filmed street corners around San Francisco, and passed the tapes around internally with a question: “If we did this at a planetary scale, what could we do?”
A researcher’s question and a founder’s instinct arrived at the same place. Maps could become the world itself.
In 2004, Luc Vincent joined Google. A French computer vision researcher, his name was known in academic circles, but he entered the Google Maps team in a modest role. When the City Block Project concluded on the Stanford side in June 2006, the technology was transferred directly inside Google. Vincent became the de facto leader.
From the start, this project had no requirements document. The spec was nothing more than “photograph the world’s roads.” The end was undefined. Google’s 20% time culture let a small, unofficial team carry an enormous ambition.
Official budget came in October 2005. Just before that, Vincent had shown an end-to-end demo at an internal tech talk — from data acquisition through processing to publication. With interns and a handful of volunteers, he built something that actually worked: processing San Francisco data and displaying it as panels on a map. Not a proposal, a working system. An engineering VP decided to join on the spot, and official project status was approved in October.
Design
The greatest challenge in decision-making was that the goal wasn’t fixed. When they said “photograph the world’s roads,” there was no internal consensus on what exactly to build, when it would be done, or what would happen after the photographing was complete. Vincent’s design philosophy was “move first, think later.” Camera hardware would be upgraded incrementally by generation; coverage decisions about where to shoot next would be made project by project. He compensated for the absence of a defined goal with organizational structure and decision-making rhythm.
There were three concrete design constraints.
First, roads never end. The world has 44 million kilometers of roads, and new ones are added every year. Old roads change with construction. If “photograph everything” is the goal, the project is permanently unfinished.
Second, camera positioning touches legal gray zones. Photographing from public roads is generally legal in the US, but not so in Europe and Asia. The distance between homes and roads, fence heights, and residential layouts vary by country.
Third, images inevitably capture faces and license plates. Publishing them as-is would trigger immediate lawsuits.
Vincent’s team chose a hardware-flexible design. The first prototype, R2, mounted multiple fisheye lenses on a car roof. The next, R5, added a laser scanner to also capture depth information. R7 packed 15 custom-developed high-resolution CCD sensors into a single housing.
The shooting platform was equally flexible. Roads where cars couldn’t go were covered by trikes. Mountain trails by Trekker backpacks (1.2 meters tall, 20 kilograms). Snowy peaks by snowmobiles. Rivers by boats. Later, cameras even went inside the ISS.
Not “design all viewpoints from the start” but “build the next vehicle when we need it.”
The decision to capture full 360-degree panoramas also paid off later. No need to decide framing at the time of shooting. Whatever direction a user might want to look afterward, that image already existed.
Faces and license plates would be handled by automated blurring. Google advertised “99% accuracy.” What that remaining 1% would mean wasn’t yet a problem.
Execution
On May 25, 2007, Street View launched. San Francisco, New York, Las Vegas, Miami, Denver. Just five cities.
Within hours of launch, a strange game began online. People searched for their own homes. They found their own cars, themselves, acquaintances in the images. A woman sunbathing. A man in front of an AA meeting building. Patrons leaving an adult entertainment venue. Thousands of deletion requests arrived in a single day.
Google ramped up hardware and rushed to develop automated face blurring. In May 2008, it began testing in Manhattan. The algorithm was designed to detect and blur human faces across the entire database. In practice, over-detection also occurred. The Dalai Lama’s face on a roadside sign was erased. Celebrity faces on movie posters were blurred. Even so, Google prioritized deployment speed over precision. On June 10, roughly one month after testing began, face blurring was rolled out to all 40 cities then in coverage. The decision was to simultaneously deploy worldwide on the premise that “99% is workable.”
International expansion proceeded in parallel. On July 2, 2008, as the first European entry, all 21 stages of the Tour de France route were published — from the Eiffel Tower to mountain passes in a single move. On August 4, 28 cities including Australia and Japan were added. On March 19, 2009, 25 cities in the UK. Filming vehicles were converted popular local models — Holden Astra in Australia, Opel in Europe. In one year, the number of covered cities multiplied tenfold and filmed distance ballooned to millions of kilometers. Filming used locally hired drivers operating GPS-linked auto-shooting vehicles; Vincent’s team never visited the field. Routes, filming schedules, and image QA pipelines were managed remotely.
In August 2008, Street View launched in Japan. Problems erupted in a different form. The cameras were mounted too high, capturing laundry and futons over fences. Japanese homes are close to roads; roads are narrow; fences are low. Camera heights designed for American suburban housing failed on Tokyo back streets. Google lowered cameras 16 inches (about 40 centimeters) and re-photographed all of Japan. The initial footage was entirely discarded. Scrapping and reshuffling all data for one country was painful in both cost and shooting time, but without meeting local standards the rest of Asia expansion would stall. Vincent’s team made that scope-cutting decision within six months.
In 2009, the Swiss Federal Data Protection Commissioner filed a lawsuit. “Automatic blurring is not 100%. Around shelters, prisons, women’s facilities, and hospitals, Google is publishing to the world images that could expose domestic violence survivors or HIV patients who happened to appear.” Google’s response: threatening to remove Street View from all of Switzerland. The bluff didn’t work.
In 2011, the Swiss Federal Supreme Court issued its ruling. In essence:
99% is not enough.
Around sensitive facilities, Google must manually verify all images in addition to automated technology.
And in May 2010, the worst scandal of all came to light. When a German regulatory authority pressed for details during an inquiry, investigators found that Street View vehicles had been intercepting and storing Wi-Fi “payload” data while driving — for three years, since 2007. The intent had been to collect SSIDs only, but unencrypted communications had been saved in their entirety. 30 countries, approximately 600 GB total. Email addresses, URLs, passwords, text messages. The contents were itemized in a later FCC investigation.
Google’s initial explanation: “A single engineer (Engineer Doe) inserted unreviewed code. The organization didn’t know.”
CEO Eric Schmidt, at the Zeitgeist conference four days after the revelation, said: “Who was harmed? Show me the person.” That same year on CNN’s “Parker Spitzer,” when asked about people who didn’t want Street View cars photographing their homes, he said, “Just move.” The comment went viral; Schmidt later apologized, saying he had “clearly misspoken.” Google published a formal blog post outlining preventive measures and established a new internal privacy audit process.
A subsequent FCC investigation revealed that multiple engineers and managers had been aware of the code’s existence. Engineer Doe’s identity was Marius Milner — the man who created NetStumbler, the world’s first application for “driving around searching for Wi-Fi networks” (wardriving). “A lone rogue” didn’t explain the hiring choice.
France, Germany, and Spain demanded Google hand over the collected data, and hard drives were ultimately surrendered to each government. In 2013, Google settled a class action with 37 US states for $7 million; in 2019, it paid $13 million in a Wi-Spy settlement. In Germany, filming was wrapped up in 20 cities in 2010 and the service effectively withdrew afterward.
People
Larry Page was 30 in 2003. Google’s IPO was still a year away. Unlike Kennedy declaring a moon landing “within a decade,” Page started an experiment with his own car. A co-founder before becoming CEO has the freedom to set things in motion with his own hands.
Marc Levoy, born 1953, was a 48-year-old Stanford professor when City Block Project began. He held the VMware Founders Professorship in computer graphics research. The Stanford research project proceeded as something like a “university-side Google internal venture.” Levoy himself later moved to Google to lead the Pixel camera team, creating HDR+ and Night Sight. City Block Project was also an experiment in deliberately dissolving the boundary between research and product.
Luc Vincent had no intention of becoming the “father of Street View” when he joined. As a computer vision researcher, he was writing foundational image processing libraries. He inherited the team because no other engineer was available to translate Page’s idea into implementation.
Vincent stayed at Google for 12 years. At the peak around 2010-2012, his team had swelled to hundreds of people. He himself left in the latter half of the 2010s and later became Chief Technology Officer of Hayden AI. When he left the project, Street View had already become an enormous operational apparatus that no single person could stop.
Marius Milner stayed out of the spotlight. Even after the FCC report identified “Engineer Doe” as him, he gave no media interviews. Whether the Wi-Fi payload collection was his decision alone or an implicit organizational agreement remains largely unrecorded in his own words.
What the Swiss Federal Supreme Court illustrated was a simple structure. In a project with a massive denominator, satisfying “99%” as a technical specification is no comfort to the remaining 1%. That’s why, around sensitive facilities, manual verification must supplement automated technology.
Legacy
Street View changed how Earth’s surface looks. 100 countries. 22 billion images. 10 million miles. That’s 42 times the distance to the Moon (380,000 km).
The unintended byproducts may be even larger.
Google Maps itself gained a competitive edge. When Apple started Maps, not having a Street View equivalent was a significant handicap.
When Waymo was developing autonomous driving, the “3D point cloud data of the world’s roads” that Street View had accumulated over a decade served as the foundation. Sebastian Thrun, who later led Waymo, built a self-driving prototype with a 360-degree camera in 2007 and personally participated in early Street View filming. Autonomous driving research produced the mapping project, and the mapping project fed back into autonomous driving itself.
For one generation of users, Street View became a device for seeing “places that no longer exist.” A home destroyed by fire. A town erased by a tsunami. A garden where a deceased parent once stood. In 2014, Google added the “Time Machine” feature, allowing users to go back through past footage.
In 2022, Google announced Immersive View for Street View’s 15th anniversary — combining aerial photography and Street View imagery through machine learning to seamlessly connect bird’s-eye views with ground-level angles. The original project’s design philosophy of “capture all directions” paid off here too.
The use cases that Page, Levoy, and Vincent hadn’t imagined in 2003 probably number more than ten.
Lesson: “Rate” Goals Get Judged by “Absolute Numbers”
Google declared “blurring faces with 99% accuracy.” As a technical specification, that’s reasonable. At the time, 99% was an ambitious figure for computer vision.
Swiss authorities found this “99%” insufficient.
The two sides aren’t contradicting each other. They’re just looking through different lenses.
Google was thinking in “rates.” 1 in 100 is within acceptable range.
The authorities were thinking in “absolute numbers.” If a billion images are captured in Switzerland, 1% is 10 million. If among those 10 million there’s one person photographed in front of a domestic violence shelter, that person could be killed.
In small-scale projects, these two perspectives are close together. 1% of 100 photos is 1 photo; the answer doesn’t change much whether you argue in “rate” or “absolute number” terms. As scale increases, the two diverge dramatically. The same “1%” becomes the boundary between acceptable noise and socially unacceptable violence.
Similar structures appear in other projects.
Boeing 737 MAX’s MCAS may have been “99.9% safe.” But in a fleet of 4,000 aircraft operating worldwide, a 0.1% systematic defect killed 346 people in two crashes.
The autonomous vehicle safety debate follows the same pattern. The relative metric of “safer than humans” is useless against the question of how many people will die in absolute numbers.
In scaling projects, the KPI design stage must decide whether to target “rate” or “absolute number.” If you keep “rate” as the goal while only growing the scale, at some point you’ll get hit with “absolute numbers” from the outside. Social and legal tolerance limits are set by numbers, not rates.
Google received this lesson, in the form of lawsuits and court rulings, when the number of images captured reached the billions.
When your project’s KPI is a “rate,” is someone calculating what that means in absolute numbers at 10x scale? At 100x?
Sources
- Computer History Museum, “Going Places: A History of Surrogate Travel and Google Maps with Street View” (https://computerhistory.org/blog/going-places-a-history-of-google-maps-with-street-view/). Stanford City Block Project and Luc Vincent’s hiring background.
- TechCrunch, “Inside Google Street View: From Larry Page’s Car To The Depths Of The Grand Canyon” (https://techcrunch.com/2013/03/08/inside-google-street-view-from-larry-pages-car-to-the-depths-of-the-grand-canyon/). Larry Page’s 2003 car-mounted camera experiment.
- Trek View, “A Brief History of Google’s Street View Cameras” (https://www.trekview.org/blog/history-of-google-street-view-cameras/). Hardware evolution of R2/R5/R7 and Trekker.
- Wikipedia, “Google Street View” (https://en.wikipedia.org/wiki/Google_Street_View). Launch years, coverage, international rollout.
- Monash University Law Review, “The Google Street View Wi-Fi Scandal and its Repercussions for Privacy” (https://www.monash.edu/__data/assets/pdf_file/0011/141230/vol-39-3-burdon-and-mckillop.pdf). Wi-Spy incident and legal analysis.
- The Register, “Google pays out $13m to make Wi-Spy scandal go away” (https://www.theregister.com/2019/07/23/google_wispy_payout/). Wi-Fi lawsuit settlement amount.
- PCWorld, “Google Agrees to Street View Re-do for Japan and Greece” (https://www.pcworld.com/article/529030/google_agrees_to_street_view_re_do_for_japan_and_greece.html). Japan re-shoot and camera height adjustment.
- PCWorld, “Swiss Court Says Google’s Street View Breaks Privacy Rules” (https://www.pcworld.com/article/224266/article.html). Swiss Supreme Court ruling.
- TechCrunch, “Google Releases Full Report On Street View Investigation, Finds That Staff Knew About Wi-Fi Sniffing” (https://techcrunch.com/2012/04/28/543181/). Marius Milner identification and organizational awareness.
- CBC News, “Google starts blurring faces on Street View” (https://www.cbc.ca/news/science/google-starts-blurring-faces-on-street-view-1.768139). May 2008 Manhattan test through June 10 full-city rollout.
- Search Engine Watch, “Schmidt’s Solution to Street View Privacy Concerns? ‘Just Move’” (https://www.searchenginewatch.com/2010/10/26/schmidts-solution-to-street-view-privacy-concerns-just-move/). Eric Schmidt’s CNN comment and apology.
- Grokipedia, “Google Street View coverage” (https://grokipedia.com/page/google_street_view_coverage). 2008 Tour de France European debut; Australia/Japan/UK rollout timing and city counts.
