Skip to content
This repository has been archived by the owner on Apr 5, 2024. It is now read-only.

Do sound as well #45

Open
imrehg opened this issue Nov 1, 2019 · 32 comments
Open

Do sound as well #45

imrehg opened this issue Nov 1, 2019 · 32 comments
Labels
enhancement New feature or request

Comments

@imrehg
Copy link

imrehg commented Nov 1, 2019

When a microphone is present.

See: https://forums.balena.io/t/balenaos-startup/37794/39

@mbalamat mbalamat added the enhancement New feature or request label Mar 3, 2020
@webtop
Copy link
Contributor

webtop commented Mar 31, 2020

I figured I'd take a look at this as I'll be doing a fair amount of camera work over the next couple of months. Right off the bat there'll need to be some reconfig as the current config only pulls back a single track from the MediaStream, the video one.

I'll be using a Logitech C920 for testing.

I'll be happy to create a GitHub public repo and commit regularly for whomever is interested. :)

@mbalamat
Copy link
Contributor

mbalamat commented Apr 1, 2020

@webtop we are open for PRs, we would love your contribution to this project.

@hinzendo27
Copy link

April 23, 2020

Hello:

Is anyone still looking at this thread? I'm attempting to make Balena-Cam operate under a better framerate. The existing one is very very slow. ??

Brian Wayne

@webtop
Copy link
Contributor

webtop commented Apr 23, 2020

I'm still poking at it. I have the refresh rate much better, but ran into an issue that was a bug in Python 3.5, which is what the project Dockerfile pulls from. The issue has been resolve now, but the Python version needs increasing.

First of all, for audio, you need to get rid of using OpenCV, as that is for video only. I'm using PyAV (already used partially), but that is how the bug was found.

@mbalamat Is there a version of fincm3 image that uses a newer version of Python?

I'll post my changes so far to my GitHub repo, for anyone interested. Audio is almost there now.

@webtop
Copy link
Contributor

webtop commented Apr 23, 2020

@hinzendo27 If you'd like to collaborate and list/fix changes together, let me know. That way we aren't both doing the same work. :)

@hinzendo27
Copy link

hinzendo27 commented Apr 23, 2020 via email

@hinzendo27
Copy link

hinzendo27 commented Apr 23, 2020 via email

@mbalamat
Copy link
Contributor

Hi guys great to hear that you like the project and want to contribute! @webtop what python version do you need? I can try bumping the python version for you though the only blocker that I had and didn't bump the version in past was the library dependencies. @hinzendo27 awesome idea about the frame rate increase can you explain to me how do you plan on increasing the frame rate?

@hinzendo27
Copy link

hinzendo27 commented Apr 24, 2020 via email

@pallsopp
Copy link

@mbalamat Anything >= 3.5.8 should be good as that's when the buf got fixed.

@mbalamat
Copy link
Contributor

@hinzendo27 to start tinkering with the code, you can try run balena-cam locally on your computer. Just install python 3. In our dockerfile there are the dependencies you need. See the pip install part. @pallsopp whenever I get some time I'll try myself to use another version. For now can you try changing the base image from what we currently have to something from here https://hub.docker.com/r/balenalib/raspberrypi3-python and regarding the tags something from here https://hub.docker.com/r/balenalib/raspberrypi3-python/tags?page=1&name=3.7?

@hinzendo27
Copy link

hinzendo27 commented Apr 24, 2020 via email

@mbalamat
Copy link
Contributor

@hinzendo27 sure no problem! Just post your question here or better in our forums here > https://forums.balena.io/ . I think in our forums the communication would be better. Because here there were a couple of times I lost the notifications. cc @pallsopp

@pallsopp
Copy link

More than happy to help out with any explanations and questions. Don't hold back, and definately use the forums because the community there is great. And yes, cc me and I'll be sure to get a notification.

@hinzendo27
Copy link

hinzendo27 commented Apr 24, 2020 via email

@pallsopp
Copy link

The technology behind all of this is called WebRTC. You can check it out on webrtc.org
The WebRTC APIs are JavaScript based, with bindings for Python. That delay at the start is partly due to the technology handshaking between the front-end and back-end. That's where the security comes into it.
There are some great videos on YouTube also concerning WebRTC, that are definitely worth a look.

@hinzendo27
Copy link

hinzendo27 commented Apr 24, 2020 via email

@pallsopp
Copy link

No problem.Just to be clear, in case it wasn't obvious, webtop and pallsopp are the same person....me. This is my work account, and webtop is my personal account. During the day I'm usually logged in under my work account, and in the evening, my personal account. Sometimes I forget to switch :)

@mbalamat
Copy link
Contributor

@pallsopp yep I noticed, thanks for clarifying though!

@pallsopp
Copy link

@hinzendo27 The basics of a Dockerfile contain everything that's needed to build a system-on-the-fly.
The FROM parts pulls a system image that someone has built and contains all the software you need to run a project. That system image contains your system files, folders, and libraries: almost a complete linux system, but just enough to make everything work.
The RUN parts are your command as if you were logged into the system: for example apt-get install commands to install the libraries you need for the project to work.
Once everything is in place, you COPY your code folder to a path on the system, and set that as your working directory.

This really isn't the place for a detailed explanation, but I thought I'd fill you in on the most important parts:
FROM an image you get your system
RUN your command to tailor the system to your needs
COPY your code to a place you can run it on the system

Hope that helps.

@hinzendo27
Copy link

hinzendo27 commented Apr 25, 2020 via email

@hinzendo27
Copy link

hinzendo27 commented Apr 27, 2020 via email

@pallsopp
Copy link

If your raspis are running containers that are on the cloud, you should be able to use balena ssh. If you are using ssh from the commandline, you might need to change the port to 22222, which is the ssh port to a local instance. Give that a try, or let me know more info about how the containers are.

Thx

@pallsopp
Copy link

FYI: I work from home and work online, so I'm always available :)

@hinzendo27
Copy link

hinzendo27 commented Apr 28, 2020 via email

@pallsopp
Copy link

Why don't we take this to the forums, and I'll be able to help you some more as this is getting off-topic for its current location.

Thx

@hinzendo27
Copy link

hinzendo27 commented Apr 28, 2020 via email

@hinzendo27
Copy link

hinzendo27 commented Apr 30, 2020 via email

@pallsopp
Copy link

What does your push command look like?

@hinzendo27
Copy link

hinzendo27 commented Apr 30, 2020 via email

@hinzendo27
Copy link

hinzendo27 commented Apr 30, 2020 via email

@pallsopp
Copy link

pallsopp commented May 1, 2020 via email

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants