notes - pulseaudio

Links: PulseAudio configuration :: PulseAudio examples :: Kodi PulseAudio guide

---

For 5-6 years, I avoided PulseAudio by sending any hooks to /dev/null. Zoom was my recent undoing -- Zoom's settings were so complicated, that it was much less timely to install PulseAudio than circumvent it. Still, instead of the unplanned kludge of PulseAudio on top of ALSA on top of OSS; instead of the elaborate trouble-shooting and configuration nightmares a nested audio 'system' brings; why couldn't OSS simply be fully developed? It was close to completion when abandoned for ALSA. And, when something so irrational as a 3-tier audio system is developed, reasonable ppl might assume the RIAA/MPAA and their IP attorneys (the DCMA Force) were involved. Anyway... that said, PulseAudio has at least improved. So now here we are.

source v. sink

We all understand the one-way graphic below, but just a note to keep in mind: these are relative.

For a person recording her voice to her lapop, she is the ultimate source, and the SSD sector is the ultimate sink, but intermediate stops can also be defined. The microphone is a sink from the source voice, but the microphone is a source for the Audio-Digital Converter sink, and so on down the line until we get to the final sink, or final destination, the SSD. When configuring, know what the software defines as a source or sink at that point in the process.

Other nomenclature isssues concern the difference between cards, profiles, indexes, sources, and outputs.

sorting for configuration

Three things to keep in mind: 1) if you know how to change underlying ALSA and/or OSS, these are still preferred, 2) failing this, well-defined PulseAudio configuration files can usually overcome and accomplish, 3) go to /etc/pulse/ for the two PulseAudio configuration files: default.pa, and client.conf. Default is just what it sounds like, so it's not the place to put all ones configurations, only the default settings for when PulseAudio starts. Client is the larger file with all profiles and so on. If one runs the PulseAudio daemon, there's also a configuration file for that: /etc/pulse/daemon.conf.

Here are a few (of 50+) PulseAudio commands used during configuration...

• pacmd list-cards
• pacmd list-sources
• pacmd list-sinks

These commands return overwhelming information and only a small portion are used in PulseAudio configuration files. Which ones? What nomenclature matters: sink, index, name, card, etc,? If I use the command...

$ pacmd list-cards

...in order to find the names to use for configuration, I will receive intimidating amounts of information.

Only the two circled items are needed for use in configuration, however which is better?

symbolic-name alsa_card.pci-0000_00_1f.3	the preferred parameter for PulseAudio configuration files.
card-index 0	unpreferred parameter inside PulseAudio configuration files. The card-index value can change across boots.

In short, I have one sound card, known to the system as alsa_card.pci-0000_00_1f.3, and this is what I should use to create configuration customizations. The good news is that I can increase the number of "sinks" on the card so that ffmpeg can access them separately or together through PulseAudio.

configuration

Go to /etc/pulse/ for the two main configuration files: default.pa, and client.conf. The overarching theme is to have well-configured (or 'well-defined') "sources", which we can then easily select from for recording or muxing. We'll need to add sinks.

Multiple Audio Collection (11:29) Kris Occhipinti, 2017. How to do multiple audio sources.

Utilizing the results from pacmd list-source-outputs, we can create recording commands.

$ ffmpeg -f pulse -i 0 -map '0' foo.mp3

adding sinks

Links Adding sinks ::

Adding sinks is important for having sources and outputs available to mux.

google cloud services initialization (cloud, colab, sites, dns, ai)

Some of Google's web services have their own names but are tied together with GCP (Google Cloud Platform) and/or some specific GCP API. GCP is at the top, but a person can enter through lesser services and be unaware of the larger picture. Again, GCP is at the top, but then come related but semi-independent services, Colab, Sites, AI. In turn, each of these might rely on just a GCP API, or be under another service. For example, Colab is tied into GCP, but a person can get started in it through Drive, without knowing its larger role. When a person's trying to budget, it's a wide landscape to understand exactly for what they are being charged, and under which service.

Static Google Cloud site (9:51) Stuffed Box, 2019. Starting at the top and getting a simple static (no database) website rolling. One must already have purchased a DNS name.
Hosting on Cloud via a Colab project (30:32) Chris Titus Tech, 2019. This is a bit dated, so prices have come up, but it shows how it's done.
Hosting a Pre-packed Moodle stack (9:51) Arpit Argawal, 2020. Shows the value of a notebook VM in Colab
Hosting on Cloud via a Colab project (30:32) Chris Titus Tech, 2019. This is a bit dated, so prices have come up, but it shows how it's done.

Google's iPython front-end Colab takes the Jupyter concept one-further, placing configuration and computing on a web platform. Customers don't have to configure an environment on their laptop, everything runs in the Google-sphere, and there are several API's (and TensorFlow) that Google makes available.

During 2020, the documentationi was a little sparse, so I made a post here, but now there are more vids and it's easier to see how we might have different notebooks running on different servers, across various Google products. This could also include something where we want to run a stack, eg for a Moodle. If all this seems arcane, don't forget we can host traditionally through Google Domains. What's going to be interesting is how blockchain changes the database concept in something like Moodle. Currently, blockchain is mostly for smart contract and DAPPs.

Colab

Notebooks are created, ran, and saved via the Drive menu, or go directly to colab.research.google.com. Users don't need a Google Cloud account to use Colab. Easiest way to access Colab is to connect it to one's Drive account, where it will save files anyway. Open Drive, click on the "+" sign to create a new file and go to down to "More". Connect Colab and, from then on, Colab notebooks can be created and accessed straight from Drive.

There's a lot you can do with a basic Colab account, if you have a good enough internet connection to keep up with it. The Pro version is another conversation. I often prefer to strengthen Colab projects by adding interactivity with Cloud.

GUI Creation in Google Colab (21:31) AI Science, 2020. Basics for opening a project and getting it operational.
Blender in Colab (15:28) Micro Singularity, 2020. Inadvertently explains an immense amount about Colab, Python, and Blender.

Colab and Google Cloud

Suppose one writes Python for Colab that needed to call a Google API at some point. Or suppose a person wanted to run a notebook on a VM that they customized? These are the two added strengths of adding Cloud: 1) make a VM (website) to craft a browser project, 2) add Google API calls. Google Cloud requires a credit card.

Cloud and Colab can be run separately, but fusing them is good in some cases. Gaining an understanding of the process allows users to know when to rely on either Colab or Google Cloud or interdependently.

Colab vs. Google Cloud (9:51) Arpit Argawal, 2020. Shows the value of a notebook VM in Colab
Hosting on Cloud via a Colab project (30:32) Chris Titus Tech, 2019. This is a bit dated, so prices have come up, but it shows how it's done.

Note the Google Cloud platform homepage above. The menu on the left is richer than the one in the Colab screenshot higher above. We run the risk of being charged for some of these features so that Google will display potential estimated charges before we submit our requests to use Google API's.

API credentials

We might want to make API calls to Cloud's API's. Say that a Colab notebook requires a Google API call, say to send some text for translation to another language. The user switches to their Cloud account and selects the Google API for translation. Google gives them an estimate of what calls to that API will cost. The user accepts the estimate, and then Google provides the API JSON credentials, which are then pasted into their Colab notebook. When the Colab notebook runs, it can then make the calls to the Google API. Protect such credentials because we don't want others to use them against our credit card.

Cloud account VM details

In the case of running notebooks and you update something, did it update on your machine or googles. its more clear on Google Cloud.

API dependencies

When a person first opens a Colab notebook, it's on a Google server, and the path for the installed Python is typically /usr/local/lib/python/[version]. So I start writing code cells, and importing and updating API dependencies. Google will update all the dependencies on the virtual machine it creates for your project on the server. ALLEGEDLY.

Suppose I want to use google-cloud-texttospeech. Then the way to updates its dependencies (supposedly):

% pip install --upgrade google-cloud-texttospeech

Users can observe all the file updates necessary for the API, unless they add the "-quiet" flag to suppress it. However, no matter that this process is undertaken, when the API itself is called, there can be dependency version problems between Python and iPython.

Note that in the case above the code exits with a "ContextualVersionConflict" listing a 1.16 version detected in the given folder. (BTW, this folder is on the virtual machine, not one's home system). Yet the initial upgrade command AND a subsequent "freeze" command show the installed version as 1.22. How can we possibly clear this since Google has told itself that 1.22 is installed, but the API detects version 1.16? Why are they looking in different folders? Where are they?

problem: restart the runtime

Python imports, iPython does not (page) Stack Overflow, 2013. Notes this is a problem with sys.path.

You'd think of course that there's a problem with sys.path, and to obviate *that* problem, I now explicitly import the sys and make sure of the path in first two commands...

import sys
sys.path.append('/usr/local/lib/python3.6/dist-packages/')

... in spite of the fact these are probably being fully accomplished by Colab. No, the real problem, undocumented anywhere I could find, is that one simply has to restart the runtime after updating the files. Apparently, if the runtime is not restarted, the newer version stamp is not reread into the API.

what persists and where is it?

basic checklist

• login to colab.
• select the desired project.
• run the update cell
• restart the runtime
• proceed to subsequent cells
• gather any rendered output files from the folder icon to the left (/content folder). Alternatively, one can hard code it into the script so that they transfer to one's system:

  from google.colab import files
  files.download("data.csv")

  There might also be a way to have these sent to Google Drive instead.

keyframes

Links: keyframe :: (wikipedia) keyframe is an "I" frame -- intraframe. types of frames :: (wikipedia) I, P, & B frames.

These blog posts are application oriented. Sometimes however, a little theory helps the application. Simply stated,keyframes are a type of i-frame manually added by a user editing a video, increasing the total i-frame count of a video.

i-frames

Video cameras record a chronological procession of pictures, frame by frame. They do so with 3 types of frames: i, p, and b. The i-frames are complete pictures, like a photo, of what is being recorded. As the camera records, it takes an i-frame and then several P or B frames, and then another i-frame, and so on. The P or B frames refer to an associated I-frame to complete the picture during playback. The Wikipedia graphic below showes the sequence. The i,p, b-frame schema was created to save memory space.

Most video devices insert i-frames about every 8 seconds or every 240 frames or so (I shoot mostly 30fps) when recording video. Newer codecs set these intervals dynamically: shorter intervals when action increases, and longer intervals when action decreases. H264 comes to mind.

keyframes

When software users edit video effects, say a dissolve transition, their editing software adds an i-frame to the beginning and end of the effect. These post-recording, user-added i-frames are in addition to the existing i-frames embedded by their camera during recording. Only these post-recording, user inserted i-frames are properly termed "keyframes".

Further nomenclature confusion can arise when software uses proprietary terms for key or i-frame intervals. For example, in ffmpeg, i-frame intervals are called GOP's "Groups of Pictures", and without regard to whether they are between key or i-frames.

raw video analysis

When I import uncut clips, I first-off detect and save all the i-frame time stamps to a file I can refer to when editing. If it's a simple edit without transition, and all my cuts are at i-frames, I might not need to add keyframes and re-encode the video. How do I get a list of i-frame time stamps? Suppose I have a clip, "foo.mp4".

$ ffprobe -loglevel error -skip_frame nokey -select_streams v:0 -show_entries frame=pkt_pts_time -of csv=print_section=0 foo.mp4 >iframes_foo.txt 2>&1

The output intervals will be timestamps, which we can easily see by catting the file.

$ cat iframes_foo.txt
0.000000
4.771433
10.410400
18.752067
27.093733
...

To determine the exact frame number (always an integer) of the i-frame, multiply the time stamp by the recording FPS. I can determine the FPS using a basic $ ffprobe foo.mp4. In this example, it revealed a 29.97 FPS. So...

29.97 x 4.771 = 142.99 or frame 143.
29.97 x 10.4104 = 311.99 or frame 312.
29.97 x 18.7521 = 561.99 or frame 562.

...and so on. We could write a short program to calulate this or export it into Excel/Gnumeric/Sheets. But this is for only a single clip and of course I want this information for each of my clips.

repair - ffmpeg

Sometimes, the keyframes become unmanageable for some reason and need to be cleaned. Typically re-encoding is required. But there is a methodology.