Crowd estimates for Hong Kong’s large pro-democracy protests have been a point of contention for years. The organizers and the police often release vastly divergent estimates. This year’s annual pro-democracy protest on Monday, July 1, was no different. Organizers announced 550,000 people attended; the police said 190,000 people were there at the peak.
But for the first time in the march’s history, a group of researchers combined artificial intelligence and manual counting techniques to estimate the size of the crowd, concluding that 265,000 people marched.
The high density of the crowd and the moving nature of these protests make estimating the turnout very challenging. For more than a decade, groups have stationed teams along the route and manually counted the rate of people passing through to derive the total number of participants. Though the use of A.I. does not make the calculation definitive, the technology helps produce a more precise estimate because it uses computers to try to count every person.
Since 2003, Paul Yip, a social sciences professor at Hong Kong University, has been producing a count of the size of protests held annually on July 1, the anniversary of Hong Kong’s 1997 handover from Britain to China. With the hopes of creating a more robust estimate this year, Mr. Yip teamed up with Edwin Chow from Texas State University and Raymond Wong from C&R Wise AI, a local technology company, to use artificial intelligence to count the crowd at the march.
How A.I. Can Be Used to Count Crowds
Using open source software, The New York Times developed a computer model to illustrate how artificial intelligence could be used to recognize people and objects moving within a video.
Analyzing a short video clip recorded on Monday, The Times’s model tried to detect people based on color and shape, and then tracked the figures as they moved across the screen. This method helps avoid double counting because the crowd generally flowed in one direction.
Accurately detecting and tracking a moving object in such a high-density condition is very difficult. Overlapping of people in the crowd and obstructions like umbrellas, protest signs and large backpacks can often confuse the A.I. system.
On the day of the protest, Mr. Yip and the A.I. team used technology that is much more advanced. They spent weeks training their program to improve its accuracy in analyzing crowd imagery.
How A.I. Was Used to Count the July 1 Protest
On Monday, the A.I. team attached seven iPads to two major footbridges along the march route. Volunteers doing manual counts were also stationed next to the cameras, to help verify the computer count.
The team’s software ran multiple models with different parameters to track people in the crowd. People could be counted when they crossed a so-called counting line within the frame of the video. Throughout the day, the team monitored the results and adapted to changing variables like the density of people, the speed of flow and different lighting conditions.
“It’s all based on visual cues, such as the brightness value, color, shape, geometry of the pixels, all tied together,” said Mr. Wong, explaining how their system distinguished human figures from objects like umbrellas.
The team began exploring the possibility of doing a count with artificial intelligence in June, when protests in Hong Kong were gaining momentum. The newspaper Ming Pao and Cable News, both based in Hong Kong, provided additional support to the team.
“What we deserve is a more accurate, more precise, and more verifiable number,” said Mr. Yip.
Ahead of the demonstration on July 1, Mr. Yip and the A.I. team tested their software at other marches and crowded places to train and fine-tune the A.I. model developed by Mr. Wong, and to test out the best angles to shoot from. They did a trial run on June 16, another day of big protest in Hong Kong, when hundreds of thousands of people marched along the same route from Victoria Park to the government headquarters.
These trials helped them prepare for issues like obstruction and overlapping bodies in the crowd and gave them ideas on how to do a comprehensive count on July 1. For example, the team decided to train their system to detect umbrellas because they are common in Hong Kong protests.
Mr. Wong also rewrote the program to run on iPads rather than industrial computers, in order to scale up the operation. The team was then able to set up the devices at more locations. They also had several spares ready to be used in case there were significant numbers of people spilling over onto nearby streets, as happened on June 16.