REST API with webcam

[duthopi]

Hi, I was wondering whether there is a way to use the API serving mode to run a model that needs my local webcam as input image? I saw the examples with uploading and receiving a .jpg image from the API, but am wondering if there is a way to send a continuous stream from my webcam?

I am first trying to run the simple flask app found here: https://github.com/log0/video_streaming_with_flask_example. However this does not output anything when I run 'floyd run --mode serve'. I believe it cannot return anything in the url because it cannot access my webcam in the first place, but I am unsure how to feed it as input.

Thanks!

[redeipirati]

Hi @duthopi,

However this does not output anything when I run 'floyd run --mode serve'. I believe it cannot return anything in the url because it cannot access my webcam in the first place, but I am unsure how to feed it as input.

The code you've tried is not working because the backend (the server code) will try to get access to the local camera (this will work locally but not on a remote machine). You have to rewrite the code in this way:

• Remove the camera acquisition code from the backend (flask code): this will have only to received and elaborate a stream of video frames
• Write a script to launch on your local machine that will get access to your camera and send the frames to the remote REST API

Let me know if you need more help.

[duthopi]

Thank you for your help.
What would be the best way to send the frames from my local machine to the floyd REST API then? Is that something I should do with node.js for example?

[redeipirati]

You can use whatever language you want. Here's a nice example from SO where it shows how to send the captured frames to a REST API with OpencCV & python.

Hope that helps.

[duthopi]

Thanks, I finally managed to get a working script to send frames and capture them in a flask server. However the scripts seem to work fine when I run them from my IDE (Pycharm), but once on the floyd serve API it throws the following error on every POST request that I send:

2019-03-24 14:54:37 PSTuwsgi 81.247.121.24, 10.0.1.101 - [24/Mar/2019:21:54:37 +0000] "POST /video_get HTTP/1.1" [200] RT:0 SZ:80 BSZ:2 [python-requests/2.21.0] REF:-
2019-03-24 14:54:37 PST2019/03/24 21:54:37 [error] 53#53: *29 readv() failed (104: Connection reset by peer) while reading upstream, client: 10.0.0.14, server: , request: "POST /video_get HTTP/1.1", upstream: "uwsgi://unix:///tmp/serve_uwsgi.sock:", host: "ip-10-0-0-203.us-west-2.compute.internal:28738"
2019-03-24 14:54:37 PSTnginx 10.0.0.14 - [24/Mar/2019:21:54:37 +0000] "POST /video_get HTTP/1.1" [200] RT:0.212 SZ:154 BSZ:0 ["python-requests/2.21.0"] REF:"-" "-" "81.247.121.24, 10.0.1.101"indent preformatted text by 4 spaces

Do you know what could be the reason that this happens on the cloud server but not when testing on the IDE? Thanks for your support

[houqp]

What's the size of your post request? We have a hard limit on how many bytes you can post to a serving endpoint

[duthopi]

It is a continuous stream of .jpeg images from my webcam, so I don't think each one on it's own would be a big request, though I will have to check this evening how big.
In total it can be a large amount of images, but each in a separate request so I guess that does not impact the size limit?

[duthopi]

I checked the image size, and each post request containing a .jpeg is only about ~75kb big. I thought that would be small enough.
However, testing with a simple text request like 'Hello world', I can successfully send it through (albeit at a surprisingly low rate of ~4 requests per seconds?). Can it be that jpegs are too large to send via http requests? How can I communicate with the server in real-time by sending a stream of jpeg images then?

Thanks!

edit: I found the following, which seems to be the same issue, and was resolved by modifying a parameter on nginx, maybe that helps?

Issue: Issue with nginx (readv() failed (104: Connection reset by peer) while reading upstream)