Stevens, an NCAA Division III university, has a new connection to the NBA: computer scientist and assistant professor Xinchao Wang.
With a focus in computer vision and machine learning, Wang has been interested in artificial intelligence ever since his time as an undergraduate. Computer vision is a field of artificial intelligence that “trains computers to interpret and understand the visual world using digital images from cameras and videos.” In computer vision, the goal is for machines to be able to analyze and understand the content of an image or video.
“When I started my Ph.D., I didn’t know that in 10 years AI would be so popular. My peers told me that if I did computer vision, I most likely wouldn’t get a job,” said Wang.
Wang went forth with computer vision for his Ph.D. studies, focusing mainly on tracking, or detecting certain moving objects over time using cameras. The cameras record videos, the goal being to track individuals and objects while analyzing the behavior of those people and/or objects.
After Wang came to the United States for his postdoc at the University of Illinois, he became interested in machine learning, which is what he works on here at Stevens. He teaches two courses, one called Machine Learning: Fundamentals and Applications, and the other called Computer Vision.
Wang first contributed to the field of computer vision while completing his Ph.D. at the École Polytechnique Fédérale de Lausanne, Switzerland. There, he helped design and prototype the NBA’s current AI system that tracks and analyzes the movements and interactions of basketball games.
“[Lausanne] is very close to Geneva, and Geneva is actually where the International Basketball Federation (FIBA) is located. That’s how my Ph.D. supervisor got a project with FIBA. We developed a system that is able to track players on the basketball court while tracking the trajectory of the ball at the same time. It can also analyze the interaction of the players and the ball, for example, at what time the player receives the ball, what time the player throws the ball, etc. With these statistics, the system acts as a tool for coaches, referees, and players to enhance their skills or to analyze the opposing team and their tactics.”
What is different about Wang’s work is the aspect of interaction; his thesis was the first to explicitly attempt to track both the ball and the player while also tracking the overall interaction between the players and the ball. Instead of independent tracking, Wang proposes seeing the big picture: “Previously people were treating [elements] independently. But if you see them all together, their interactions provide you with statistics.”
To make the system work, Wang aims a camera at each corner of the court. After the cameras are set up, they are calibrated, mapping the relative position of each camera. Then, by running AI and computer vision algorithms, they can recover the 3D coordinates of all the players and the ball for each time instant. “When you have the location of each player at each time instant, you can sort of link them into full trajectories; you recognize their behaviors,” said Wang.
The most recent computer vision project Wang has worked on is 3D pose estimation. Instead of tracking trajectory, pose estimation focuses more on finer details. It uses cameras and video to recover and understand the pose of a person through time, whether the person is jumping, raising their hand, opening their arms, and so on. This is important in the case of civilian use: “for example, at the airport you are not allowed to leave your luggage unattended. The system can detect when a person puts down baggage and leaves,” Wang said.
Nowadays, Wang has mostly been working on interpretable machine learning. By itself, machine learning is the study of algorithms that computer systems use to “perform a specific task without using explicit instructions, relying on patterns and inference instead.” Machine learning, and more specifically deep learning, are “producing the best results in almost all the applications of computer vision and AI: language processing, speech analysis, video analysis, image analysis, medical imaging, whatever you can imagine,” Wang says.
Big data is one of the driving factors for the success of deep learning. Computer scientists feed big data into deep learning models so that they can “train” them.
Currently, people see deep learning as a black box; while deep learning systems are successful at tasks like identifying objects in images, “how they do so remains largely a mystery.” The way in which deep learning models work are “shielded from human eyes, buried in layers of computations, making it hard to diagnose errors or biases.” Interpretable machine learning, however, aims to analyze what is happening within this black box by providing explanations for deep learning models’ outputs.
“You can sort of imagine this black box as some sort of intelligent box. We want to analyze what kind of knowledge is contained within each of these black boxes, and transfer such knowledge from one black box to another,” Wang says. This is known as transfer learning.
“Why do we bother to understand what is happening here? Because once we have an understanding of what is happening within blackboxes, we can provide a guarantee of its ability. If we understand it, we know its behavior will be logical, and we will not be afraid to deploy them in the world,” Wang explains.
When explaining the differences between machine learning and computer vision, Wang notes that “machine learning is more theoretical—you deal with probability and matrices. Computer vision is more application based.” The success of computer vision projects depend on defining the problem, and then coming up with robust, reliable systems that can solve the problem. Wang’s NBA tracking system provided a commercialized solution for coaches and players. In machine learning, success depends on defining optimal theories, models, approaches, and methodologies so that they can be applied to different applications (one application being computer vision).
Explaining his passion for artificial intelligence and computer science in general, Wang touched upon his interest in science-fiction movies as a child. “When I was young, I was fascinated with science-fiction movies […] even today I still watch Marvel movies,” Wang noted with a big smile. “When I was younger I loved E.T.”
“[Later,] when I was in primary school, I started programming. I was in China and the teacher there was teaching a programming language called Logo. I was 9 years old,” Wang said. “It was very fundamental programming; you have some sort of pointer, and some sort of canvas, and you ask the pointer to go straight, turn left, turn right […] you had to do it slowly, with very simple comments. I began to play with it, and I was fascinated. Then, for my undergraduate, I chose CS.”
Throughout his education, Wang noted that his supervisors are the ones who influenced him the most in regard to his career. Dacheng Tao, his undergraduate supervisor, is currently a professor of computer science at the University of Sydney in Australia. Tao “is the one who really guided me to do research in AI when I was an undergraduate. At the time he suggested I do research in facial recognition.” At that time, in 2008, facial recognition was not yet mature. “He showed me the project and I thought it was very interesting, so I started to do research with him when I was in my senior year of undergrad. He’s the guy that told me ‘okay, this is the way to do it’ in terms of AI.”
Wang’s Ph.D. supervisor, Pascal Fua, taught him how to conduct “scientific research, and how to write good papers so that normal readers can understand […] so that people can follow your work.”
Finally, Wang’s postdoc supervisor, Thomas Huang, is a research professor at the University of Illinois. One of the first leading figures in computer vision, pattern recognition, and human computer interaction, Huang began to work on the idea of computer vision in the 1960s, when computer punch cards were used for programming. “He taught me that whenever you want to address a problem, you should think of long term impact […] whenever he sees a problem, he can see the intrinsic core of the problem. If you discuss some technical problem with him he will point you towards a long term direction of research that you can follow.”
“I learned almost all my knowledge from all of them. Without them, I am nobody,” said Wang.
Within the next year, Wang plans to publish a paper on his work in interpretable machine learning, unlocking the mystery of black boxes, and deep learning. When he does, his work will make an impact on this ever-growing field of research.
Be First to Comment