An Interview With Pixellot’s AI Ninja, Gal Sadeh Kingsfield

by Adam Hanft, Global Branding Expert, Strategic Advisor to Pixellot

 

I hold in my mind two images.

One is an 11-year-old giving a soccer game every fiber of his being.

The second is the image of a data scientist applying AI to every move of that young athlete.

It might seem an exercise in contradictions, but at Pixellot they live side by side. And in the case of this interview, that data scientist is Gal Sadeh Kingsfield.

Gal is a global leader in AI, having worked as a data scientist at Amazon, GE and Walmart. There he led a team focusing on computer vision, which is core to his work at Pixellot. Prior to joining us he ran AI at Perion, a leading adtech company.

Now Gal runs a team of 25 brilliant AI engineers who are innovating at the frontier edge of where AI meets youth sports.

 

AH: Okay, let’s start by telling us what AI means at Pixellot.

GSK: Artificial intelligence technology is the basis of everything we do at Pixellot. To be specific, all the content we create – which is at massive scale – is made possible by computer vision, which is the branch of AI focused on images and video. We create an automated production machine of live sports events.

There are humans on the field, but no humans in the loop.

AH: Pixellot has been around for 13 years. That’s way before ChatGPT was launched. Sam Altman was only 28 then!

GSK: Yes, Pixellot was a pioneer, and that long history gives us perspective and insight. We know what works and we know what doesn’t. It also gives us a vast repository of content. More about that later.

AH: Define massive scale for me. What is the scope of Pixellot’s programming?

GSK: It’s enormous. We create 120x the video hours of ESPN. Think about that.

AH: Pixellot is a different kind of ESPN. The “Extraordinary Scale Production Network.”

GSK: Love that! And actually, we automate what the NBA does manually. They are still tagging manually, believe it or not.

What’s more, we capture video for 20 different sports.

AH: Do you need 20 different algorithms?

GSK: Not really. There are basically two different types of games. You have play-based games like American football, volleyball, and baseball. Then you have flow-based games like soccer and basketball. So you need one algorithm for play-based and one for flow. Then you tune it.

AH: That’s the essence of automating production?

GSK: Yes, but what we do is more complex that just automating production. We create highlights, we create stories. We generate stats – even advanced ones. Think of it as athlete-centered platform for the entire experience. Coaching too.

AH: And AI is behind it all?

GSK: Every last pixel of it! Let me go a little deeper. We detect players, too. Not by their faces, we respect privacy and never do facial recognition. We are doing physical trait recognition. We’re looking at your entire physique. We’re looking at your height, your hairstyle, your jersey color and of course the number on the uniform.

AH: That’s a fascinating level of AI-personalization.

GSK: There are more visual cues than you might imagine. Movement, shoulder width, even how a player bends. Then we train a neural network to differentiate one player from another by their physical attributes. In fact, these natural attributes are exaggerated during the stress of the game itself.

AH: Seems like you’re automating what parents can naturally do, which is pick their kids out on the field, even from considerable distances. I wrote about that phenomenon for the Pixellot blog.

GSK: Yup, evolutionary biology fascinates me.

AH: You described how AI works for player identification. Talk about it for understanding individual performance within a game.

GSK: That comes next. We track everything! Take basketball. We can determine how high you jumped, how much did you ran, your maximum velocity. Even more granularly, we can say are you more efficient when you penetrate to the basket from the right or from the left, or when you stand with your back to the basket on the elbow. We can capture everything that is being tagged manually by the NCAA or NBA.

AH: How’s your accuracy?

GSK: We’re proud but humble. We’re not at 100%, we are above 90%. But the experience is just as important as the accuracy. We believe trends and measuring improvement is important. If we can say that 10 games ago, your average was 60 from the elbow, and now you’re 65, motivating. We want athletes to compete against themselves

AH: Motivation is important in youth sports. We want to use data to encourage participation, not discourage, I wrote another blog post – here I am marketing myself, again – about the participation crisis among young people.

GSK: Yes, we need to inspire. That said, if someone wants to create an in-league competition, we can show leaderboards, but that’s not where we’re aiming.

AH: How unique is this AI capability?

GSK: We’re planning on publishing a scientific paper on what we did for following players; we believe it is unprecedented in the long context window of sports tracking. Tracking someone throughout an entire game is a very different challenge than doing it for one minute or one possession or one play.

AH: Let’s turn our attention to the future. Talk about the whole new app experience you are building. Will someone be able to use an agent to create a clip of their child, either individually or stitched together? Say every successful shot that he had during the season.

GSK: Yes, it will be an easy editing process from the footage we’ve collected. At the same time, we’re giving feedback back to our models – so we can understand whether we made the wrong identification, a wrong event detection or we missed something.

AH: It seems like this could go in a million directions.

GSK: It definitely can. Behind it is something we call a storytelling engine. It’s going to be a holistic view of an athlete’s entire life. Looking ahead, our goal is to create pre-game and post- game stories that would be able to create engagement because we understand that youth sports is deeply local A small town in North Carolina cares about its high school team.

AH: At the same time, what you’re building is a real benefit for coaches, too.

GSK: It won’t be long before players and coaches will be able to chat about performance with the AI model. For example, analyze this clip, analyze this pick and roll for me. What did I do wrong here? Why is this a miss?

We will create AI advisors for coaches to converse with, and then close the loop with their athletes.

AH: What did you train the Pixellot AI on?

GSK: We indexed our knowledge base, which is vast. Tens of thousands of hours and games. No one has that. Period. Full stop. But that’s just the beginning. We’re rolling it out to test users, we’re seeing what they’re asking, we’re learning their actions. Then we’re retraining the model, rolling it out to a bigger audience, and retraining again. That’s how data science works. It’s not something that you just roll out and that’s it.

AH: Not to get too technical, but it sounds like what you’re doing is using the frontier models to create vertical AI for youth sports.

GSK: Yes and it’s going to be hyper-vertical for each sport. I alluded to this earlier. Say for example you are a multi-sport athlete. I played basketball, which I sucked at. I also played handball. But I was best at volleyball. Imagine if back then, I had an app that told me that my skills were superior at volleyball and should focus there. I woudn’t have needed the trial and error. I wouldn’t have needed to keep sucking.

AH: Let’s wrap up by talking about monetization. Local youth sports need money, right? They are always short of funding. And there’s a way to accomplish it elegantly, without destroying the experience. I called it “soft monetization” in this blog post I wrote.

GSK: Yes, absolutely. We’ll be able to introduce advertising in away that’s not going to feel too intrusive. It will feel organic. We are detecting moments within the game where we can naturally insert a message. We will reduce the view to 66%. And then we’ll serve an ad that’s relevant to the age group that’s watching.

In the app it will be like an Instagram story, a familiar interface that includes messages between clips, as you scroll.

AH: Anything else you want to say that we didn’t get to?

GSK: We are bringing countries and markets of all sizes a youth and amateur sports experience that is automated, personalized and permanent. We are doing this via an app and an SDK that will cover the world’s most popular sports. Getting kids to play is important work. Sports teach discipline, resilience, collaboration and self-improvement. We are proud to be part of this effort. It is an example of AI at its best.