xgamma -gamma 1.500

xrandr | grep " connected" | cut -f1 -d " "

xrandr --output STRINGFROM_ABOVE --mode 1360x768

/ after some inactivity display brightness drops and we see the lines /

to send keys and mouse to active app:

; disable auto updates: https://linuxconfig.org/disable-automatic-updates-on-ubuntu-20-04-focal-fossa-linux

; TIP: list these by gsettings list-recursively | grep -i "suspend\|sleep"

; set desktop background to gray (note black will reflect off of screen)

gsettings set org.gnome.desktop.background primary-color '

888888'

gsettings set org.gnome.desktop.lockdown disable-lock-screen 'true'

; wakelock issues at 4 minutes the screen resolution gets shitty

gsettings set org.gnome.settings-daemon.plugins.power ambient-enabled false

gsettings set org.gnome.settings-daemon.plugins.power idle-dim false

gsettings set org.gnome.desktop.session idle-delay 0

; Disable Notification: "Computer will suspend very soon because of inactivity"

gsettings set org.gnome.settings-daemon.plugins.power sleep-inactive-battery-type 'nothing'

• control computer

• external webcams

• samson microphone

• workcomputer

* internal microphones and speakers.

* cover internal cam

* Settings

* brightness=minimal

* True Tone: No

* Battery

* Turn off display after = never

Media Production Notes

• safety measure

* never try customizating or changing things during a live run. the audio feedback saves.

* vicinity problem

* vicinity can be tracked using location services or wifi, so even devices without a gps can be geo tracked.

• Sample effects supported

* background noise cancellation

* when a surprised is introduced

* situational

* character development

• Testing

• Color and Lighting

• Geometry

* laptop: head to frame ratio of 1/4-3/16 Width, 1/2 - 1/3 is the standard for laptop shooting.

• Anomation

* mechanical devices (chair) create smoother animations and more predicable picture quality.

* Hello

* Smile

* Scratch Face, Head

* Look away

* Come in and Sit

* Waive to people

* Point at camera

Smoke Tests

• Audio: Dial into user, then on device play the AHJ audio

* ZM > Settings > Test Audio: Should hit middle green bars

• Video: Join an empty ZM session on Device

* Turning off VC should show transparency (no OBS)

• Full: Join empty ZM session on Device and play the AHJ audio

* The sync loop should respond

Testing

npm run testvideo

workstationaudioout@peer is not available yet

clear && printf '\e[3J';npm run testservice queryObject.service xcast@xcast,portforwarding@xcast,mixeradmin@peer,audiooutput@peer,jsonio@peer,virtualcamera@peer

Creating Audio/Video Pipelines

There are several recommended tools available for this task:

• ffmpeg/ffplay
• vlc
• gstreamer: to consume mpeg4 content refer to how-to-install-the-mpeg-4-aac-decoder-and-the-h-264-decoder:

sudo apt-get install ubuntu-restricted-extras;

The oldest protocol for streaming is RTSP and RTMP. They have a few seconds delay and are not web friendly. The quickest protocol is WebRTC. For an in depth analysis refer to wowza-streaming-protocols.

RTSP

docker run --rm -it -e RTSP_PROTOCOLS=tcp -p 8554:8554 -p 1935:1935 -p 8888:8888 aler9/rtsp-simple-server

export RTSPURL=rtsp://localhost:8554/mystream

To publish content to the RTSP service

/ on linux /

ffmpeg -f avfoundation -framerate 15 -video_size 640x480 -i 1:0.0 -f rtsp rtsp://192.168.1.12:8554/mystream

vlc $RTSPURL

gst-launch-1.0 -v playbin uri=$RTSPURL uridecodebin0::source::latency=300

To consume the content:

/ to show on desktop /

/ to route to video device /

Non RTSP alternatives

• MPEG-TS (MPEGTS) streams using UDP push: You can use ffplay to render

/ ffplay will listen for udp packets /

ffmpeg -f avfoundation -list_devices true -i ""

• rtp audio stream

ffplay rtp://127.0.0.1:1234

• WebRTC: it is more complex and requires more setup options. you would need to use a webrtc server

* spreed: https://www.fosslinux.com/45051/how-to-install-spreed-webrtc-server-on-ubuntu.htm

• Low-Latency HLS (LL-HLS): See this article https://www.wowza.com/blog/hls-vs-webrtc

Using a hardware input camera

v4l2-ctl --list-devices

HD Webcam C615 (usb-....):

/dev/video3

Linux Virtual Camera

/ v4l2loopback is the virtual camera on linux /

/ make sure the user is part of the video group /

sudo rmmod v4l2loopback;sudo modprobe v4l2loopback videonr=0 exclusivecaps=1 card_label="vcam0";

sudo v4l2-ctl -d0 --list-formats-ext

sudo rmmod v4l2loopback;sudo modprobe v4l2loopback devices=2 videonr=0,1 exclusivecaps=1,1 card_label="vcam0,vcam1";

/ This is needed or python will not work /

/ Do not use cheese or ffplay. They cause trouble. Use gst instead /

You can then pipe various sources to virtual camera device using any of toolchains described above. We recommend using gst:

gst-launch-1.0 ximagesrc startx=0 starty=0 endx=640 endy=480 use-damage=0 ! videoconvert ! v4l2sink device=/dev/video0

If you like to use ffmpeg see ffmpeg-send-video-to-dev-video0-on-ubuntu:

ffmpeg -f x11grab -framerate 15 -video_size 640x480 -i :0.0 -f v4l2 /dev/video0

If you are interest making the opencv python output available as a virtual camera source to the system refer to virtual-camera-for-opencv-using-v4l2loopback.

GStreamer Cookbook

gst-inspect-1.0 faceblur

That plug-in come with

apt list --installed | grep gstreamer

FFMPEG/PLAY Cookbook

/ generate test videos /

ffplay testsrc.mpg

~/ffmpeg -i recording.mov -vcodec h264 -acodec mp2 recording.mp4

/ extract audio /

~/ffmpeg -f avfoundation -list_devices true -i ""

/ save output of device 1 to test-output.aiff /

~/ffmpeg -f avfoundation -video_size 640x480 -framerate 30 -i "0:0" -vcodec libx264 -preset veryfast ~/tmp/raw.mp4

/*

trim, cut the middle of the video

*/

export FFMPEGINPUTVIDEO=~/izyware/izy-idman-tools/id/$IZYANIMATEUSERID/virtual/izyanimate/$IZYANIMATE_WORKSPACE/datasamples/1/video.mp4

[0:v]split=2[copy1][copy2],

[copy2]trim=112:120,setpts=PTS-STARTPTS[part2],

~/ffmpeg -i ~/izyware/izy-idman-tools/id/$IZYANIMATEUSERID/virtual/izyanimate/$IZYANIMATEWORKSPACE/datasamples/1/video.mp4 -q:a 0 -map a ~/izyware/izy-idman-tools/id/$IZYANIMATEUSERID/virtual/izyanimate/$IZYANIMATEWORKSPACE/datasamples/1/audio.wav

/ audio conversion /

~/ffmpeg -i data/test-44.1-16-mono.wav -ar 48000 data/test-48.0-16-mono.wav

~/ffmpeg -i audio/test-48.0-16.wav -acodec libvorbis audio/test-48.0-16.ogg

~/ffmpeg -i file.jpeg source.png

Working with Animate Workspace Tools

./ppm.sh "analyze?analyzeVideo" --queryObject.imageRepoPath=$izyAnimateWorkspacePath/frames--queryObject.inputVideoCapturePath=~/izyware/izy-idman-tools/id/$IZYANIMATEUSERID/virtual/izyanimate/$IZYANIMATE_WORKSPACE/datasamples/izyanimate-1641837127-w880xh580.mp4

./ppm.sh "animate?animate" --queryObject.imageRepoPath=$IZYANIMATEROOT/frames/$IZYANIMATEWORKSPACE--queryObject.inputVideoCapturePath=$IZYANIMATEROOT/datasamples/$IZYANIMATEWORKSPACE.mp4--queryObject.maxFrames=10000--queryObject.frameSize=400--queryObject.waitForKeyboardOnEachFrame=True

./ppm.sh "animate?animate" --queryObject.imageRepoPath=$IZYANIMATEROOT/frames/$IZYANIMATEWORKSPACE--queryObject.maxFrames=10000--queryObject.inputVideoCapturePath=2--queryObject.frameSize=400--queryObject.cropInput=True--queryObject.resultsPath=$IZYANIMATEROOT/results/$IZYANIMATEWORKSPACE

To list audio devices

npm run getaudiodevices

For dependencies:

• MacOS users, the naudiodon has a dependency on node-gyp which will require command line tools to be installed. so make sure to do:

xcode-select --install

Platform Considerations

• We do not recommend the Apple M1 Family:

* M1 family does not have display scaling to 1280x720 and will slow down the system.

Camera Resolution

In contrast, smartphones have camera with much higher resolution and quality. Screen recordings can be megapixels (like 2000x3000).

As of 2021, the virtual camera provided in broadcasting software offers @1280x720 resolution / 30 FPS.

External Resources

• github
• npmjs

NOTE

Known Issues

• SKP

* does not respond when in the background for incoming request

• CLI

* it shoud always be running and it should automatically recycle after hours.

* simple tcp socket streaming to an input using the node audio lib

• Video

* Face landmarks detection - Opencv with Python

* Deepface Lab: from iperov

* Paper - Face2Face - Real time Face Capture and Reenactment of RGB Videos

* period check for making sure tracks are in sync (they tend to get out of sync)

* ability to have more than 2 tracks for environment switch (happy, leave, etc.)

• information bucket

* add "pings" to information bucket under a bucket and rename it to "acvitibyt"

* can track across different devices (device id is 1 parameter)

• camera broadcast tool

* show username in foreground in white or show logo (so that it looks like Zoom)

* npm this: pkill -f firefox;declare -x DISPLAY=":0";firefox "http://....."

• audio

ChangeLog

V7.1

• 7100002: add quickRefresh method for remote desktop
• 7100001: improve remote desktop and refresh algorithm for displaycrops

V7.0

• 7000037: implement delete for izyidman/5/pkgdashboard/tab/rawdata

* without this the app could not be shared to non shadow users which would cut off access to value and even if the app was shared, we would have to hide none manual payload grabs and device logs for non shadow users which would be difficult.

• 7000036: virtualcamera: reposition cropped area
• 7000035: selenium: bug handing forever - connect to the UI and manually
• 7000034: remotedesktop: stop refreshing when hidden
• 7000033: service/kinesis implement yOffset

• 7000032: service/kinesis/macos/nodec - update to only wobble the mouse pointer

• 7000031: improve silence detection algorithm by extending the time window for looking at silent frames
• 7000030: break up stopStreamWhenPowerIsBelow to isSilentWhenPowerIsBelow and stopStreamingWhenSilent

* Move the features away from audioInput to socketWriterNode since script processor does not allow sending buffer of length 1, so cant do it as onaudioprocess

• 7000029: xcastmonitory - cleanup and seperate focus function from refresh
• 7000028: restore enforceSampleRate functionality
• 7000027: mixeradmin - mediaStream tracks need to be stopped to clear the recording icon in the tab
• 7000026: mixeradmin - destroyAllAudio when network disconnects or errors

* otherwise, mic audio will keep streaming into a closed socket and will generate errors

• 7000025: mixeradmin - reenable remote desktop and allow toggle
• 7000024: mixeradmin - add network status to qos panel, move metrics calc into the service.
• 7000023: mixeradmin - implement reset functionality

* introduce new state 'localclose' for modelling user initiated closure

• 7000022: net/websocket - generate error event when the browser does not call onclose on failed conenction

• 7000021: net/websocket - send onConnectionState with state=error on bad address

• 7000020: mixeradmin - allow updating config
• 7000019: improve ui code quality and user experience

* use //service/serviceName?method to query values on events. do not pass state around in the events

• 7000018: implement resume button for unblocking paused audio contexts
• 7000017: improve connection and audio stream monitoring and networking

* capture issues related to streams in the audiovisualizer view

• 7000016: use service notification framework to couple the visualizer to resources. implement autoPlayPolicyWarning. implement beepAndPauseOnAudioInit for speakerContext
• 7000015: webAudio - return the first device found regardless of name. Safari/iOS does not include Default in name
• 7000014: service/mixeradmin/peer - fix microphone code

* use notifySubscribers and service subscription model

* improve Webaudio to allow parity between browser native code and non browser environment

• 7000013: service/mixeradmin/peer - fix metadatastr collection for speaker

* temporarily disable currentMicSilenceDetected

• 7000012: audiovisualizer - add startDelay

* allows audioContext to get created and be ready

• 7000011: service/mixeradmin/peer - improve speaker code. move speaker visualizer out of selection widget.
• 7000010: service/mixeradmin/peer - improve cleanup cycle.
• 7000009: Webaudio - implement useNativeAudioContext

* allows mix mode deployments between browser and nonBrowser envirnments

• 7000008: switch the services to service?onConfig pattern
• 7000007: utilize service?compose+config pattern for testing
• 7000006: extract xcastSourceMonitor/peer from xcastSourceMonitor/api
• 7000005: remove references to shim-izy-proxy
• 7000004: move service/mixeradmin/peer to dashboard code. more infrastructure updates.

* temporary disable stopStreamWhenPowerIsBelow, clearExistingObjects functionality

* added documentation for externalScriptProcessorObject

• 7000003: cleanup and use izy-proxy monitoring infrastructure. update module lifecycles by adding forcemodulereload when needed.
• 7000002: move lib/monitoring to izy-proxy. use izy-proxy monitoring infrastructure. update service configuration definition for vatar.

* use pkgModuleString and full config names

• 7000002: freeze V5. lockdown izy-proxy and frontend changes.
• 7000001: cli - use newChainAsync feature and move setup out of the main loop

V6.9

• 6900007: Give refresh option when disconnected
• 6900006: dashboard, increase delayThresholdMS
• 6900005: dashboard, capture user inputs in uri

* on mobile we often disconnect. this feature will make refreshing easier

• 6900004: bug fix swith to whereClauseForAllVisibleCollaboratorsBasedOnGroups
• 6900003: adjust scaling for iOS devices
• 6900002: use stopStreamWhenPowerIsBelow on the mic. Do not send samples when quite.
• 6900001: improve streamproto1 event handlers. add silenceDetected to the user mic. implement adminMicStatus string

* string usage will implementing cli easier

V6.8

• 6800019: migrate userinput/audioinput to audiostatus panel

* decouples cloud settings from audio settings

• 6800018: migrate userinput/audiooutput to audiostatus panel

* decouples cloud settings from audio settings

• 6800017: send only keepAlive packets when detecting silence in the input audio

* stopStreamWhenPowerIsBelow property is implemented for nonBrowserAudioInputNode and is configurable based on the volume

* customer issues addressed:

• 6800016: selenium - run the service on userspace port after izy-proxy upgrade
• 6800015: audio - improve monitoring and thresholds
• 6800014: qos - add audio visualizer and glancable status for audio streams

• 6800013: remotedesktop - single select user when mouse click on user desktop

• 6800012: remotedesktop - organize keyboard and refresh into inputOptions. add minimal design

• 6800011: remotedesktop - improve refresh rate
• 6800010: remotedesktop - adjust layouts. add new shortcuts
• 6800009: remotedesktop - allow single or multiuser selection fronend
• 6800008: remotedesktop - implement multiuser frontend api
• 6800007: add autoRefresh option

• 6800006: implement ability to hide selection portions of ui
• 6800005: kinesis - implement waitForUIUpdateMS

* improve ui responsiveness for inputless monitoring

• 6800004: kinesis - optimize slack view.
• 6800004: selenium - improve title logging. fix select all selector ember65
• 6800003: add total uptime metric
• 6800002: add labeling and colors for mic metrics
• 6800001: add labeling and colors for speaker metrics

V6.7

• 6700011: add enforceSampleRate to audio input

* security feature for when the device is non standard

• 6700010: implement adjustSampleBufferSizeToNewFrequency with dynamicResampleRate

* useful for when the data source has hardware bugs. For example GHW USB AUDIO would not sample at 48khz and would return a stream at 48.1khz.

• 6700009: lib/webAudio - bring scriptProcessor on par with W3C Web Audio API
• 6700008: kinesis - escape input string
• 6700007: kinesis - dismiss cert warning fix coordinates
• 6700006: qos- add netRate for admin speaker
• 6700005: audioinput- implement crashAfterTimeSeconds

* this can be utilized as a workaround for the memory leak in the workstationaudioout. customers had reported memory overnight going to 6Gig.

• 6700004: audiooutput - clear up legacy code. use IzyScriptNode with getWrapper functionality. add currentSpeakerBufferDepthInFloat32Samples metric.

• 6700003: add test scripts for adminmictoworkstationaudioin.sh and workstationaudioouttoadminspeaker.sh
• 6700002: capture onConnectionStateOutcome and display to qos. add aggregatorStatus to qos. improve metadata string exchange mechanism for protco1. implement sendKeepAlive for socketWriter. add consoleMode to service

* keepAlive useful for no source is connected but the QOS metrics need to be send. this will make the feedback independent of workstationaudioout and debugging easier.

* add useAudioCopy mode for legacy workstationaudioout clients

• 6700001: dfl - engineConfig.appDisplayCropConfig

* allows the user to customize which part of the display to project

V6.6

• 6600009: audiooutput - implement skipScriptNodeForSocketToSpeakerData

* allows better performance for non-browser environments

• 6600008: audioinput - improve diagnostic beep.
• 6600007: mixeradmin - improve logging for cleanupExistingConnection
• 6600006: monitoring - add compact day-time timestamps to logs
• 6600006: virtualcamera - allow using kernel device as the video source vs. rtsp

• 6600005: implement service/selenium
• 6600004: audiooutput - unfocus the user when peer disconnects

* without this feature, the admin will still see that the user is "focused" and will assume that the audio is getting through which is incorrect. the delay will start increasing, but it would be unclear why.

• 6600003: audiooutput - implement beepAndPauseOnAudioInit

* provides smoke test for audio filters

• 6600002: dfl - add modprobe command to startup sequence

* ensures that service can be started from a reboot with no issues

• 6600001: portAudio - provide workaround for the blocking input capture when the device is the system default on MacOs

V6.5

• 6500020: virtualcamera - show user warning message
• 6500019: dfl - map dflConfig.videoInput to /dev/video0
• 6500018: useAudioCopy - fix the bug where deviceId of zero is not recognized
• 6500017: cli legacy - fix bug in remoteaccess/synccode.sh where it fails when plat folder is missing
• 6500016: cli legacy - allow remoteaccess/runapp.sh to specify config

• 6500015: webAudio - use deviceNameGrepStr for selecting audioinput
• 6500014: virtualcamera - implement video source from rtsp for dfl
• 6500013: virtualcamera - implement dfl engine
• 6500012: virtualcamera - allow engineConfig and gstPipelineConfig
• 6500011: virtualcamera.directvideo - scale video to full screen size, implement feedcamerartsp

service/virtualcamera/feedcamerartsp.sh ubuntu-xxx

• 6500010: virtualcamera - implement directvideo engine
• 6500009: improve admin cli and add hot keys for fast user switching
• 6500008: implement admincli for alternative to the browser interface

npm run admincli queryObject.queryObjectId ../queryObject.xcast.xcast.json queryObject.focus

• 6500007: implement QOS visualizer component

• 6500006: implement destroyAudioDevice and clean up devices when socket disconnects.
• 6500005: xcast - implement cleanupExistingConnection to customize admin reconnects
• 6500004: networking - allow turning off reconnect for sockets. fix nodejs websocket client code.
• 6500003: Implement QOSOutputMetrics for node outputs to improve performance monitoring. Fix the input double start bug.
• 6500002: improve convertArrayBufferToFloat32 and add support for 32BitPCM
• 6500001: implement outputDirectlyViaFFPlay

* useful for bypassing naudio interface and debugging networking performance issues

V6.4

• 6400016: admin dashboard - improve QOS visualization
• 6400015: disconnect and restart xcast when multiple admins try to connect

• 6400014: fix timestamp encryption algorithm by using Floaty64 datatype
• 6400013: implement streamproto1 for audioinput.

* implement filters for audioStreamBetweenNodesLogAllStreamData and audioStreamBetweenNodesGrepStr

* add magic number to header to detect corruptions

* implement simulateChoppyChunks to test audio transmission over low quality networks

• 6400012: admin console - use socketWriterNode to stream microphone audio

* allows for overlaying protocol metadata and performance metrics inside the audio stream

• 6400011: implement streamproto1 for sending metadata over audio stream

clear && printf '\e[3J';npm run testservice queryObject.service testmixeradminspeakerxcast@xcast,testmixeradminspeakerenduser@peer;

• 6400010: lib/monitoring - improve format and add support for debug info by extraInfoInLogs

* formatting the logs such that they are consumable from terminal

• 6400009: consolidate mixeradmin/peer and admin dashboard. implement nonBrowserDebugNode, dataStreamMode, streamProtocol and QOS metrics.

* dataStreamMode (streammodecooked, streammoderaw): raw will be useful for highperformance relaying of streamins without inspecting the contents

* enableQOSMetrics: collection and sending of QOS metrics

• 6400008: improve net/websocket lifecycle managemens, events and closing mechanism
• 6400007: admin dashboard - add shim-izy-proxy for newChain and use net/websocket and lib/globals

* enables easier testing and implementation of cli interface

• 6400006: admin dashboard - add audio visualization for output. add config for streamProtocol definition
• 6400005: move audio/lib to lib/audiosignal and use in admin dashboard. utilize audioDeviceObject in admin dashboard state management
• 6400005: admin dashboard - add audio visualization

* helps quickly determined whether the audio is connected and working

• 6400004: admin dashboard - improve keyboard input experience

* added icons and removed extra text

• 6400004: admin dashboard - move audio controls to the settings area
• 6400003: admin dashboard - move logview to the settings area
• 6400002: lib/monitoring - allow registering monitoringIngestionService

• 6400001: admin dashboard - create settings modal and move source selection to settings

V6.3

• 6300038: implement resetworkstation feature
• 6300037: implement runCmdBeforeStart to allow custom shell commands

* this will allow controlling of device volume on service startup

• 6300036: rename virtialspeaker to audioout and create workstationaudioin
• 6300035: admin dashboard - implement ctrlc
• 6300034: implement deviceNameGrepStr for IzySpeakerNode
• 6300033: useAudioCopy for audioinput as an interim solution
• 6300032: implement AudioNode disconnect. implement dataStreamType for socket and onRawSampleCB data connectivity option.

* support Float32 and 16BitPCM datastreams

• 6300031: add support for audio copy tool to act as an audioinput service

node tools/acp.js 1 :10003 0 1

• 6300030: implement IzyAudioInputNode
• 6300029: allow prefixing the app config from cli

node apps/workstationaudioout.js dev

• 6300028: fix dead socket reconnect bug. include workstationaudioout in standard test

clear && printf '\e[3J';npm run testservice queryObject.service xcast@xcast,portforwarding@xcast,mixeradmin@peer,virtualspeaker@peer,jsonio@peer,virtualcamera@peer,workstationaudioout@peer

• 6300027: fix bug where reConnect would flood the socket with requests
• 6300026: implement audio copy - tools/acp for quality assurance on the client machine

• 6300025: cleanup mixer control panel and improve usability by adding verbose mode. unified peer and front-end jsonio access by using lib/jsonio
• 6300024: rename webaudio service to mixeradmin and fix broken dashboard deps
• 6300023: dashboard - use default for sourceSelector
• 6300022: migrate izyidman/5/pkgdashboard/tab/xcast2 to izyidman/5/xcast/dashboard
• 6300021: kinesis frontend - add fullscreen mode
• 6300020: audio mixer - send empty buffers when channel is muted

* this will address the blips and audio glitches

• 6300019: implement userCmdToSendBeforeEachScreenCapture to customize the behavior
• 6300018: remova experimental features for padding the audio with silence

* the output stops when 1 of the inputs stop

* the output stops. it will also resample to the sine frequency and cause disortion.

• 6300017: move statusmonitor to xcast service
• 6300016: implement apps/workstationaudioout
• 6300015: implement audioinput service and mixWorkstationAudio mode for mixer admin audio aggregator

clear && printf '\e[3J';npm run testservice queryObject.service ajmixermixer@xcast,ajmixeradmin@peer,workstationaudioout@peer

• 6300014: add suppoer for tcp sockets to nonBrowserSocketWriterNode class
• 6300013: decouple service type from name and implement type property for service definitions to allow defining multiple services of the same type

* allows defining context specific naming for services

• 6300012: kinesis service - fix bug for findVNCSession
• 6300011: xcast kinesis ui - ability to switch between users
• 6300010: xcast ui - pass userId as parameter
• 6300009: fix bug for when ffmpegAudioInputDeviceId is zero
• 6300008: place the fifo pipes in tmp directoy
• 6300007: implement useOscillatorToCoverGaps flag for virtual speaker
• 6300006: adding -re to ffmpeg input does not make a difference
• 6300005: fix the clicks, pops and gaps noise for audio generated using aresample(libswresample) using async parameter

* utilize async and usewallclockas_timestamps

* see also this post for more information https://stackoverflow.com/questions/20439640/ffmpeg-audio-transcoding-using-libav-libraries

• 6300004: migrate virtualcamera to service oriented architecture

clear && printf '\e[3J';npm run testservice queryObject.service xcast@xcast,portforwarding@xcast,mixeradmin@peer,virtualspeaker@peer,jsonio@peer,virtualcamera@peer

• 6300003: add downx10 and upx10 to xcast frontend
• 6300002: reorganize xcast2/userinput/audioinput/api.js for consumption from serice oriented components.
• 6300001: update the test audio for xcast frontend.

V6.2

• 6200055: fix issue with xdotool mouse click by adding delay between move and click.
• 6200054: improve createWaveHeader to accept zero for totalSamples
• 6200053: implement deviceNameGrepStr for virtualspeaker
• 6200052: implement logdata feature for portforwarding
• 6200051: refaactor and cleanup jsonio
• 6200050: handle OPTIONS requests for JSONIO service
• 6200049: implement legacyAudioStream for allowing interoperability with legacy xcast audiostreams
• 6200048: improve jsonio@xcast error handling
• 6200047: improve configurability and service structure for kinesis. add support for tigervnc client, pikvm and realvnc client.

* pkvim does not support mouse over HTTP API. it is only supported over the ws streaming protocol.

• 6200046: fix bug in service jsonio/xcast
• 6200045: update runapp.sh to launch items from apps folder.

runapp.sh hostfolder kinesis

• 6200044: implement shelllisten.sh and shellreverse.sh for quick access and setup.

* make sure to setup the bash environment on the target environment.

vim ~/.bashrc

export PS1='\w>';

• 6200043: introduce connectionManagementService feature for manually handling the connection. implement portforwarding service.

* testxcast was updated to utilize portforwarding

clear && printf '\e[3J';npm run testservice queryObject.service xcast@xcast,portforwarding@xcast,mixeradmin@peer,virtualspeaker@peer,jsonio@peer

• 6200042: implement kinesis service

clear && printf '\e[3J';npm run testservice queryObject.service kinesis@xcast,kinesis@peer

• 6200041: fix bug in networking. clean up unused interfaces. new test model using testservice.

clear && printf '\e[3J';npm run testservice queryObject.service xcast@xcast,mixeradmin@peer,virtualspeaker@peer,jsonio@peer

• 6200040: migrate jsonio api to xcast/peer model. improve logging and add jsonio peer module.

* onNewConnection will return dataBinding for socket which could be set to "manual"

• 6200039: implement for virtual speaker enforceLegacyAudioServiceType

6200038: implement apps/.dev daemons

• 6200037: service launch - support servicename@context schema
• 6200036: audio/file/input - allow specifying custom path for the input file
• 6200035: implement apps/virtualspeaker.js and apps/xcast.js as nodejs daemon
• 6200034: reorganize modules lifecycle and config for improve testability. add reference config test/queryObject.xcast.json and use augmentQueryObject for config startups

clear && printf '\e[3J';npm run testxcast

• 6200033: lib/webAudio - implement new extensions and improve metrics and diagnostics collection

* nonBrowserAnalyzerNode

* implement AudioNode.setName function for tracing data

• 6200032: diagnostics: add probePCM16 to debug tools probeFloat32. Improve logging.
• 6200031: convert audio/stream to generic ffmpeg interface for audio/ffmpeg replacement
• 6200030: consolidate audioDeviceObject creation to lib/webAudio. create generate audio/stream using ffmpeg
• 6200029: lib/webAudio - implement nonBrowserSocketReaderNode
• 6200028: virtualcamera add setupMachineXSession script and move environment configs into queryObject.xcast.json
• 6200027: lib/shell -- allow sudo mode
• 6200026: userGUIBackend - sent neutural key on each screen scan

* useful for preventing the screen saver from coming on

• 6200025: client - add zoomLayout switching
• 6200024: client - improve enumerations and styles for zoom regions and shortbuts
• 6200023: client - implement izyremotecmd_ and useful keyboard shortcuts

// tip: use xev for calibration and to see exactly what needs to happen

• 6200022: client - implement GUI zooming feature

• 6200021: improve mouse and keyboard actions. use relative coordinates for scalability. handle source outages gracefully.
• 6200020: implement lib/monitoring.chain processor. improve large number formatting using formatFriendly
• 6200019: refactor client - implement apps/virtualcamera for daemon process management.
• 6200018: refactor usersGUIBackend - add X system startup scripts and make config driven. create apps/users-gui-backend for daemon process management.
• 6200017: lib/shell - implement kill method

• 6200016: xcastfe - select system default for input on startup
• 6200015: xcastfe - select system default for output on startup
• 6200014: xcast - turn off allowNonSystemDefaultSelection

• 6200014: xcast - implement bi-directional graph tracking. implement close method for cascade releasing resources
• 6200013: xcast - improve setupAudioGraph, logging and buffering. import tab/xcast2/lib/monitoring
• 6200012: implement end to end tests for audio streaming (testwebaudio)

clear && printf '\e[3J';npm run testwebaudio queryObject.verbose.forceUpToLevel 2 queryObject.verbose.transport true queryObject.action streamInputDeviceAsWave|streamWaveLoop

• 6200011: rewrite client/webaudio/net to use lib/webAudio
• 6200010: improve the webAudio shim and other libraries

* implement nonBrowserAudioBuffer (non-interleaved IEEE754 32-bit linear PCM)

* add audio/lib.convertFloat32to16BitPCM to library. implement socket

• 6200009: lib/monitoring - fix bug for counting the data volume
• 6200008: implement wave stream conversion using ffmpeg

npm run teststreamconversion queryObject.verbose.forceUpToLevel 2 queryObject.verbose.audioInputSample true queryObject.verbose.streamConversion true queryObject.sampleRate 16000

• 6200007: implement lib/webAudio universal audioContext and audioNode interfaces.
• 6200006: implement stream convertor using ffmpeg
• 6200005: codec.wav - implement createWaveHeader
• 6200004: import lib/stream. use loopStreamSamplesArray
• 6200003: create lib/stream.loopStreamSamplesArray. add .log to monitoring
• 6200002: implement sendMetadataOnAudioSwitch. Improve logging.
• 6200001: introduce DEV1000320003 connection config for implementation tasks

V6.1

• 6100046: switch to using global.applicationState.focusUserId for focus model

* storing the volume in server (not connection object) so when restarting the client for testing the no reclicking and reconfiguring is required and the user can continue where they left off.

• 6100045: implement smooth transition for loop changes

* it will act as a deboucing so that for short sentences we dont get a loopy jump.

• 6100044: virtualcamera: blur frames when not ready. refactor looper and add unit tests.

$IZYPYTHONROOT/python3 test.py

• 6100043: test/client: implement mutePattern feature

• 6100042: video - implement prototype dual bi-directional loop. remove seek command
• 6100041: fix examineAudioSignal. provide auto blurring features.
• 6100040: debug/audio - auto detect system speaker
• 6100039: add data/test-48.0-32-mono.wav and instructions
• 6100038: improve net/broadcast, use standard monitoring library. support ws://address format. introduce streamToAllUsers flag. fix bugs
• 6100037: virtualcamera - implement audio stream connector

* this is needed for feeding the audio from the virtualcamera client and having synced audio/video

• 6100036: codec/wave: use allowExperimentalWaveParser to allow format code 65534
• 6100035: improve audioDeviceObject initial sequence amd include streamProperties

* randomize the deviceId to avoid collision

• 6100034: implement clienttest cli

* useful for testing audio sync, video quality, etc.

* works as an standalone client, with no networking required

npm run clienttest queryObject.client.type virtualcamera queryObject.user.id $IZYANIMATEUSERID queryObject.user.izyAnimateWorkspacePath ~/izyware/izy-idman-tools/id/$IZYANIMATEUSERID/virtual/izyanimate/$IZYANIMATEWORKSPACE

• 6100033: create client/virtualcamera

* improve the seperation between networking and client logic layers

npm run client queryObject.address broadcastip:port queryObject.client.type virtualcamera queryObject.user.id $IZYANIMATEUSERID queryObject.user.izyAnimateWorkspacePath ~/izyware/izy-idman-tools/id/$IZYANIMATEUSERID/virtual/izyanimate/$IZYANIMATEWORKSPACE queryObject.verbose.networking true

• 6100032: cv/ fix bugs in libraries
• 6100031: refactor convert16BitPCMToFloat32. Add analyzewave command. Improve debug/audio library.

npm run analyzewave queryObject.user.id $IZYANIMATEUSERID queryObject.user.izyAnimateWorkspacePath ~/izyware/izy-idman-tools/id/$IZYANIMATEUSERID/virtual/izyanimate/$IZYANIMATEWORKSPACE

• 6100030: add ability to mute ranges in audio/file.
• 6100029: refactore audio/file and create libs for extracting audio properties
• 6100028: ipc - improve error handling for setupWriter
• 6100027: turn off video when no inputVideoCapturePath is passed
• 6100026: webaudio - increase thresholdForSendingData

• 6100025: implement GUI access via remote desktop and vnc

npm run usersGUIBackend queryObject.address $IZYANIMATESOURCEADDRESS queryObject.id $IZYANIMATEUSERID queryObject.verbose.networking true

• 6100024: findAudioDeviceIdFromName - fix bug when parsing ffmpeg output
• 6100023: use shortConnectionsStatus for saving space on mobile devices
• 6100022: reaplace deviceId with audioInputAddress.
• 6100021: add verbose.allErrors flag to always display errors regardless of verbose level
• 6100020: implement stream monitoring tool. improve logging.
• 6100019: xcast - fix connectionId memory leak

• 6100018: implement remoteaccess/download.sh
• 6100017: implement logview component for collecting metrics. optimize audio processor buffer size to avoid networking delays for audio input streaming
• 6100016: ffmpeg - calculate actual samplerate for the stream. allow looking up of device ID using device name
• 6100015: audioinput - break testaudio into smaller chunks, cleanup logs
• 6100014: xcast - return full status when jsonio focus action
• 6100013: consolidate select identity into monitoring widget
• 6100012: update configs to use izyware.com
• 6100011: wssocket - use p5/ prefix and use url to determine service type
• 6100010: webaudio - fix scoping issue
• 6100009: audio input - play test audio in a loop
• 6100008: minor tweaks to id picker and auto refresh
• 6100007: implement e2e webaudio and decouple the transport from the ws audio device.
• 6100006: implement proper error handling for readFile JSONIO
• 6100005: implement the ability to select the source server.
• 6100004: implement the ability to route audio input directly to output. embed sample data to allow networkless operation.

• 6100003: improve audio output panel ui
• 6100002: only warn on samplerate mismatch and conitnue. clear up the local and cloud config.
• 6100001: improve audio device lifecycle management. imlement destroyAudioInput for ffmpeg device and destrot when remote socket closes

V6.0

• 6000020: implement wss and webaudio services

npm run wss queryObject.verbose.networking true queryObject.address ws://localhost:20110 queryObject.verbose.audioInputSample true queryObject.verbose.audioInputLogginIntervalSeconds 2 queryObject.mode streamInputDeviceAsWave|streamWaveLoop

• 6000019: improve variable names and handling
• 6000018: lib/ipc - implement rawPipe type

* allows for connecting to binary streams from other applications using a fifo pipe

• 6000017: implement front-end piece to webaudio client type
• 6000016: implement seperate cli interface for partial content service

npm run partialcontent queryObject.address localhost:10003 queryObject.verbose.audioInputSample true queryObject.verbose.partialContent true queryObject.mode streamStaticFile queryObject.fileName test-48.0-16.wav

npm run partialcontent queryObject.address localhost:10003 queryObject.verbose.audioInputSample true queryObject.verbose.partialContent true queryObject.mode resizeAndStreamWaveFile queryObject.fileName test-48.0-16.wav queryObject.factor 5

npm run partialcontent queryObject.address localhost:10003 queryObject.verbose.audioInputSample true queryObject.verbose.partialContent true queryObject.mode streamInputDeviceAsWave

• 6000015: implement front-end ux for partial content streaming
• 6000014: implement audio output streaming using the ScriptProcessorNode
• 6000013: cli - add shell script for automating vnc access
• 6000012: implement partialcontent (406) client type and wave audiocodec

* would allow streaming audio and video into the native browser without special javascript.

• 6000011: load sample only if window.audioData is not set.
• 6000010: refactor existing wav streamer to file/input. support readFile JSONIO command.
• 6000009: add allowNonSystemDefaultSelection. Implement playing audio file from the source server and test audio feature. add UI for selecting sourceType for audio output. Implement activeAudio.onSocketMessage for streaming back from server. Add optional refresh button.
• 6000009: fix logging for ws/input. fix bugs.
• 6000008: implement identity userinput. improve UX for user switching.

* move the widget to top

• 6000007: implement generic audio streaming interface to xcast and allow echo testing.
• 6000007: improve socket connection management for xcast2
• 6000006: implement xcast2 audio capture switching, release resouces and memory leaks and add logging.
• 6000006: update xcast2 and improve the UX for audio device selection and source monitoring. add support for websockets
• 6000004: update remote access tools and add vpn and vnc scripts.
• 6000003: xcast2 - prototype websocket audio input (ws/input)
• 6000002: prototype websocket audio input (ws/input)

• 6000001: initial izy-xcast fork from izy-idman-tools xcast

V5.7

• 5700030: update README
• 5700029: fix tests
• 5700028: implement mixer1 reciever type

* uses audio detection to animate videos

• 5700027: support backwards compatibility with v1 client and make recieverType mandatory for recievers. support new client types and new the XCAST2 dashboard. include audioMetadata in the video frames.
• 5700026: make inputVideoCapturePath optional for video input and make label generation consistent with videoOutput model
• 5700025: incorporate the reciever scripts into syncdata
• 5700024: allow postFrameReciept to manipulate the displayConfig

• 5700023: displayConfig should default to display size when face property is not specified

• 5700022: remoteaccess scripts, add config/sourceinformationbucketid
• 5700021: make videoGenerationMode optional for the reciever and default to the broadcaster
• 5700020: augment metadata for video capture. improve visualization library

• 5700019: use outcomes to handle postFrameRecieptSetup
• 5700018: add delay to ipc error handler exit to allow child process error output time.
• 5700017: improve variable and filenames.
• 5700016: reorganize facepicker for better modularity
• 5700015: modularize the videogenerationmode and introduce generic applyFeatures and extractFeatures
• 5700014: implement izyProxy.ldfn using the launchString syntax

* allows using izyProxy.ldfn('path?fn') in python

• 5700013: handle IPC post video capture open. return exit code 1 on node fifo error.
• 5700012: Do face tracking and auto zoom for the first frame only. Loop playback when playing from file

• 5700011: return proper exit code when ppm.sh fails
• 5700010: default to direct when videoGenerationMode is not speced
• 5700009: improve error handling when frames have magic header

• 5700008: improve remote access scripts and allow launch without copy
• 5700007: implement remote access scripts
• 5700006: improve video recorder functionality
• 5700005: implement FOMM videoGeneration
• 5700004: implement custom videoGenerationMode types. improve labeling. add flag for downsampleInputVideoFrames.

• 5700003: add support for NVIDIA GPUs
• 5700002: automate VNC connections
• 5700001: improve testvideo messages and text

V5.6

• 5600041: add removeBackground flag for displayConfig
• 5600040: fix workspace loader bug
• 5600039: improve auto crop and zoom. allow defining videoInputConfig.json file
• 5600038: add offsetH to displayConfig
• 5600037: implement directVideo streaming for more flexibility in deployment and scenario tests

* queryObject.videoMode captures this method

• 5600036: pyjs ipc - fix readLine bug
• 5600035: add queryObject.inputVideoCapturePath to testvideo for allowing full e2e testing on the same device
• 5600035: implement autocrop, calibration, background removal. apply workspace displayConfig during record

* applying workspace displayConfig during recording will make adjustment easier

* calibration will allow normalization of feature detection and is neccessary to make animation work across different shapes and geometries

• 5600034: automatically adjust brightness. add sleep command

npm run sleep

• 5600033: add more tests
• 5600032: bug fix - update serialization algorithm
• 5600031: add more logging. improve pyjs interop system
• 5600030: bug fix - replace hardcoded id
• 5600029: vide - allow display configuration and background image
• 5600028: python integration - adjust exit codes based on verb outcome

* failed outcome will return non zero exit code.

• 5600027: audio -introduce virtual python audio output device. improve testing and audio handling.

• 5600026: utilize more environment variables and use ~/izyenvironment.sh

• 5600025: improve ipc and shellexec communication and process lifecycle management.
• 5600024: reciever - honor the queryObject.focus flag on each reconnect.

* useful for debugging when the server crashes

• 5600023: fix broken tests and typo
• 5600022: allow padding file urls as audio device (deviceId file://sample32khz16.wav)

• 5600021: streamer - fix bug with frame flow control
• 5600020: video out - kill existing python processes on startup

* this will remove lingering video windows from a previous crash

• 5600019: audio out - add queryObject.testOutputOnStartup for recievers
• 5600018: video out - allow window placement
• 5600017: implement the ability to runAs the video output, add videotesting

* useful for broadcasting applications where control and video are seperate desktops

• 5600016: python izyProxy - return empty queryObject when no cli parameters are provided
• 5600015: add integration for cv video with vatar broadcast system
• 5600014: remove ipc from animate. clean up libraries.
• 5600013: extend the broadcast frame to include metadata
• 5600012: report time in local time zone
• 5600011: add integration tests for broadcast

npm run testbroadcast

• 5600010: cleanup setupVatarToCV IPC configuration
• 5600009: integrate CV luanch into source server. add IPC.
• 5600008: cv - add lib/config
• 5600007: cv - remove dead code. update gitignore.
• 5600006: add computer vision recorder, analyzer and animator library.

* debounce the change detection and define a threshold for change.

* output averaging: remember the previous pick and allow smooth transition on updates

• 5600005: add mouse emulator.
• 5600004: add statusmonitor. implement jsonio xcast server interface
• 5600003: remove startvirtualcam from OBS cmdline
• 5600002: implement audio mixer
• 5600001: implement camera broadcast tool

* improve frame speed rate

* white background will generate glare in dark room

* automatic login

rm ../camera.zip; zip -r ../camera.zip . -x .git/\ -x node_modules/\ .