Questions: Performance on hobby courts, weird camera angles, performance on low power devices, data consistency

[tdymel]

Hey!
Amazing project! Its really nice to see that you keep working on this project.
I am personally looking forward to play around with this project and use it in the future to provide real time statistics.

The app that I am creating right now is more oriented towards hobby teams and semi professional teams. So they would play in school courts where the court may not be able to be distinguished this well and where the camera angle may be a little awkward and may not even show the full court.

How do you think will the performance be like for this use case and how can one improve the model to work well on it?

Another question: I am planning to run this model on the clients device, i.e. a smart phone. Is this model fast enough to run on these kinds of devices preferably in real time?

Another question:
What is the data like, can the model consistently label the same players the same way? Such that throughout a whole scan a recognized person would get the same label?

Thank you so much!
Cheers!

[masouduut94]

thanks for kind words.

Regarding datasets:
I have the ball detection and volleyball actions detection datasets available. but not the video classification dataset.

Realtime inference: It is possible only if you reduce the latency of the frame by frame operations with ML; for example instead of using 3 models (player detection + ball detection + action detection ), you can run 1 thorough model. This means you need dataset having annotated all of the objects of interest. I have seen such project in table tennis that the authors designed a compeletely cusomized deep learning model that could track ball as well as segmenting the players, the digits (for reading scores, and set numbers) and net movements.
check this link

On client device:
I have never done such project yet.
Doing that may bring some drawbacks:

1. models perform best if they run with powerful GPU machine. using models on mobile phones is not popular becasue drains battery. you should use some techniques like model distillation, model quantization, and etc to reduce the inference time so it is suitable for mobile phone cpus.
2. model accuracy often decreases drastically if you apply model optimization techniques like the ones above.

But if you do simpler tasks, like the Rapsodo company doing the golf ball tracking on applications. This task can be often done by doing multiple filters written with openCV and other fast libraries and don't always need deep learning.

In such projects we prefer API + client architecture, so we can send packages to server and expect the server return the reply after process.

I guess the last question, is about player tracking; meaning that you want each player get same ids throughout the game.
For this one, I would say there are techniques namely as MOT or "multi-object tracking" that can be utilized to track the objects like players. speaking of experience, I would say such techniques are good but not perfect. most of the time they need to be reset. For example in volleyball, after each score, there is a clockwise rotation between the players. knowing that helps us to reset the players location. or maybe we could use the OCR technique to detect the player jersey's number, so we can follow that and hope that we can detect that player idenetity in realtime.

[tdymel]

Hey,
thank you for getting back at me this fast.

My project is currently not at a point yet where I have time to tackle this interesting problem, but I hope to arrive at this point by Q1 2026. I would love to come back to you then and ask you for some of your insights.

I am currently building a platform for Volleyball teams to connect at events, organize games, trainings and tournaments and manage teams. On top of it I have the vision to also feature statistical analysis where this project might come in handy. Real time may be the icing on the cake, as you said one could also think about a frame queue, so bigger models could be utilized on these restricted devices. The reason I want to use smartphones is because the project runs on no money whatsoever, so one has to use every piece of free infrastructure one can get :D

My vision is to have a model that runs on a smartphone and basically segments a video in different scenarios, similar to your model, like serve -> receive, receive -> set, set -> spike etc.
At the same time it will track players and the ball and connect these actions to segments and players.
Using this data and analysis of each player and the overall team performance is generated.

After the video has been processed, the system could prompt the user to annotate the recognized players. Such system could be used to do deduplication of players. Jersey detection might also be an option, although female players hair often obscures the number and many hobby teams dont have jerseys at all.

Thanks for the link and all the insights I can gather by reading into your project source code.