No more TV ads, Machine Learning to the rescue !
Ads are everywhere, the whole digital economy is fuelled by ads, the world is a gigantic commercial.
Content is created only for enabling maximum visibility of targeted ads.
We are trapped !
Well… a bit extreme vision, hopefully we have some good solutions to eliminate most of them on the WWW:
- browser ads blocker
- personal proxy with dns filtering
- … probably a lot more, I should see some in google ads after searching “stop ads” thousand times
Saying that, let’s consider we can reasonably escape from massive ads flood on Internet. It’s a matter of few hours investment to find the suitable system for your internet consumption.
Among the alternate solutions we also have pay content like VOD, online paper subscription…
Now here’s my problem, I live in the middle of nowhere with a bandwidth around 2.5MB…
Got 2 lovely kids, nourished with Youtube and Fortnite and a wonderful wife, watching pets on facebook.
Then by default and as a good Al Bundy I consume terrestrial TV quite a lot. Can’t wait 20 seconds to watch 15 seconds of program, VOD is technically not viable with my internet connection performance.
I’m then facing ads invasion on terrestrial TV, mmmm… a bigger challenge than filtering web content…
Yes, some of you will say “time shifting”, but I’m “hopping” like crazy and even if we have only 27 free channels in France, it’s too much hardware for the purpose.
PIP ? Picture in Picture was really a great feature, but unfortunately dual tuner TVs are not very common nowadays (why ?), I’ll need to buy an extra tuner for each TV in the house, no go…
My holly grail: when I watch a program and commercial starts, TV swap automatically to any other channel, get back to the original program when ads end. Simple, isn’t it ?
- grabbing tv stream without impacting my internet bandwidth
- can’t lock an expensive computer for hosting solution
- must be portable and minimal configuration
- can be used by anyone
- must be developed by a dumb guy in less than 3 days (2 light weekends)
The clumsy solution
Device connects to a web page on the PI through the home wifi network.
Web page is showing the TV live stream from the PI, served by FFserver and proxied by Nginx. Format is mjpeg, the most universal solution I’ve found to stream video without sound and with very limited latency.
From a dedicated page, user can “teach” when the tv channel is showing ads or not. Simply by pressing a button. The model is then automatically uploaded to the PI. Timing was too short to think about good looking UI…
Tracking naughty ads
In the device browser, model is downloaded and cached in local DB. User can select the channel to monitor. The nodejs process, proxied by Nginx , is receiving the requested channel id, stop the current FFmpeg streaming, start on the new channel.
Live stream received on the page is passed for prediction to deaplearn knn model created previously.
Show big RED message when ads, back to BLUE when finished so you can continue to watch your favourite program.
- Getting TV stream without using internet bandwidth
Found a little TVHat for the raspberry PI, works like a charm but others TV tuner dongle also work very well.
- Reliable ads detection without image processing knowledge
I used a previous prototype showing how you can train and run a model in the browser thanks to deaplearn js (now tensorflowJS). Super easy, I’ve spent time only on developing the ability to export/persist the trained model. TV channels are using DOG (Digital on-screen graphic), the DOG is removed during ads to avoid hiding ads elements. Good for us, we can track this DOG, knn is doing a good job but there’s for sure better solutions, I hope… learn more here
- CORS issues
Mmmm… working on canvas to create tensor requires it’s tainted from the same origin. Simple solution, proxy everything with Nginx, basta
- Being able to infer from any device (laptop, smartphone, apple tv, chromecast…) requires proper reading of the TV stream
I don’t need audio so mjpeg (motion jpeg) is the best streaming format I’ve found, widely natively supported…
- Minimise cpu usage for re-encoding image, raspberry is a tiny insect.
FFmpeg manual compilation to support GPU acceleration, tuning, again, again and again… Cropping the image has been a good bonus, actually I just need to grab part of the screen where tv channels are putting their DOG. At the end, Raspberry CPU is showing 25% activity on average and rock solid.
- A raspberry PI (3B+ to be trendy)
- TV hat for raspberry, or any usb tuner that can work on debian
- ffmpeg && ffserver (v3.4 is the latest including ffserver)
Total cost: 50$ (PI + TV Hat).
You can lower by buying a cheaper TV dongle, seen some around 5$…
What’s still on the table
- UI and UX, it’s dramatic, will take more than 2 days to fix :-)
- automatic upload of trained model, easy but time consuming, tough to get it with my 2 days max constraint
- automatically change TV channel, difficult to get a generic solution. Got only LG TVs and LG API is good enough so it’s easy for me but need pluggable code for others :-(
- automatically detect the cropping zone to optimise the whole experience, no clue how to do that, need investigations…