Author: Anton Runov

When we got the first report that FarPlay app failed to connect after upgrading to Sequoia 15.0, we checked the logs and found that most users who upgraded hadn’t experienced issues, so we thought it might be a one-off case. But after receiving several more complaints it became clear: we had a problem.

FarPlay is an app that allows musicians to make music online interactively. This requires much lower latency than common audio/video conferencing apps provide. To make online jamming feel like you’re in the same room, end-to-end latency needs to be below 20-30ms. That’s why FarPlay relies exclusively on peer-to-peer (P2P) technology. It allows us to use the shortest and the fastest possible path to deliver audio data from one side to another. But P2P connectivity depends strongly on firewall behavior. Firewalls that are too restrictive or incorrectly configured can easily make P2P connections impossible.

Shortly after Apple released macOS Sequoia 15.0, reports of issues with different firewall and network software began popping up. Customers of large security software companies like CrowdStrike, ESET, and SentinelOne reported issues. Some VPNs stopped working too. For some FarPlay users, it looked as though their firewalls just started blocking audio traffic entirely even when their firewalls were turned off completely in system settings. At the same time, others didn’t experience any issues. I was not able to reproduce the problem on my test Mac.

We are a very small company, just a few people. If IT security giants couldn’t solve the issue, what could we do? About 60% of our users are on Mac, and even if only 10% were affected, it could have been disastrous for us. When P2P connectivity is broken, FarPlay simply can’t work. A music teacher can’t provide a scheduled lesson. Musicians can’t conduct their final rehearsal before a concert.

We began researching the changes in the new version of macOS and discovered that the new “Local Network” section in the “Privacy & Security” settings affects FarPlay connectivity. This setting has existed in iOS for a while and has now been added to macOS. Normally, it would only be required for connecting FarPlay clients within the same local network. However, we found that enabling this setting for the FarPlay app also resolved the connectivity issues for users affected by the Sequoia firewall problem.

So, if you have a networking issue with Sequoia, check out the Local Network section of Privacy & Security settings. I can’t say if this helps with other Sequoia issues, but it was a lifesaver for us. Shortly after we published this workaround on our support forum, other users confirmed that it worked for them as well.

Just recently, Apple released Sequoia bugfix version 15.0.1. As usual, its description is vague, it says it “improves compatibility with third-party security software”. We don’t have enough data yet to say if our problem was fixed with this update. Our users already had a solution at that point.

—Anton

FarPlay version 0.3.7 is out today, with the following improvements:

• Windows: support for native audio devices (experimental).
• Windows ASIO: use ASIO Control Panel for ASIO buffer size configuration.
• Linux: support for Jack and PulseAudio devices (experimental).
• Select input channels directly from channel menu on the main session window.
• Preferences: improved channel configuration.
• Use fixed -3 dB gain for recording mix tracks.
• New BO channel layouts for mixing local and remote audio (with fixed -3 dB gain).
• Fix an issue when mono mix was played in stereo when monitoring your own sound.
• Extend volume sliders to +10 dB. Double clicking on the slider sets it to 0 dB.
• Improved HiRes displays support on Mac and Windows. Respect Windows system scaling settings.
• Minor UI fixes (don’t open multiple preferences windows, error messages, tooltips).
• Connectivity workaround for rare cases when FarPlay keeps displaying “Initializing service…” message (SSL CA workaround).
• Connectivity fix: never send packets larger than standard Ethernet MTU (1500 bytes). Prevents audio disconnects right after session join (a rare issue happened mostly on Windows at specific configurations).
• Use fixed 1000 ms delay when recording without BO (ignore BO delay in this case).

Have you ever thought that the teaching of music, not to mention music in general, was one of the fields most damaged by COVID lockdowns? Closed venues and cancelled concerts were just the tip of the iceberg. Musicians lost not just the ability to perform but also the ability to teach, rehearse, and in many cases even make music at all. There are lots of things we learned how to do remotely. You can hold staff standups, business meetings or teach math through Zoom. But not play music.

If you are surprised with this statement, you are not alone. It was a huge surprise for many musicians when they started searching for tools for remote collaboration in 2020. They just found it was impossible at all to keep time while playing remotely. Why? The short answer is latency. Delivering audio signals from one end to another takes time. We can tolerate it when we speak (even though it is the cause of a lot of awkward moments in video calls). But for playing music it is a killer issue. There were a number of research works showing that the maximum one-way latency that musicians can tolerate without losing time sync is about 30-50 ms. This is the time that a sound wave takes to travel a distance of 10-15 meters. This is why large orchestras need a conductor. Without visual gestures serving as synchronization signals, it would be impossible for musicians spread over an area of this (or larger) size to keep time. But the minimum latency available with video conferencing apps is about 100 ms, 2-3 times higher than the threshold. And this is for the most perfect network conditions and close geographic locations. In more typical cases it goes up to 200-400 ms and beyond.

The physical distance plays a critical role too. The speed of signal propagation in networks (no matter whether they’re wire or fiber) is close to the speed of light (~300 km/ms). It seems quite high but bear in mind that IP network routes consist of many intermediate legs between hops. Each I/O operation implies extra delay which nearly doubles the ideal theoretical latency. As an example, a typical round-trip ping time between Europe and the East Coast of the US is about 100 ms which means adding an extra 50 ms to the one-way latency.

In the spring of 2020 I got a call from my friend, a sound engineer. “Anton, musicians desperately need low latency audio conferencing”. I spent years grappling with latency during my career in IP telephony. I did know a lot about the difficulties and insurmountable hurdles for this. I said “forget it man, it’s impossible”. But we started searching around and eventually found that in the early 2000s Chris Chafe at Stanford University started a project called JackTrip aimed to address these issues. The idea was to remove as much extra latency as possible from the communication path by optimizing I/O, buffering and signal processing. You cannot change the speed of light or global network architecture. But there is still quite a bit of room for optimization. JackTrip made it possible to play rhythmic music over the Internet over distances up to 500-1000 km.

The one issue with JackTrip was its complexity. It was written for research, not for most musicians. It had no user interface, just a UNIX-style command line. It used direct IP address-port connections to the remote peer which means you have to configure your network router to open ports. It was based on Jackd audio server for internal audio I/O which is a standard tool on Linux, but setting it up on Mac or Windows feels like a nightmare even for professionals. Very few musicians were able to overcome all the difficulties and start using JackTrip for their music.

One of the early adopters and enthusiasts of JackTrip was Dan Tepfer. An NYC based jazz pianist and composer, he was also doing several projects combining music and technology (you might want to check out his Natural Machines to see). Since spring of 2020 he has used JackTrip for his weekly Monday Livestreams to play remote duos and trios with other musicians.

I started working with Chris and Dan in the summer of 2020. In cooperation with Dan, we developed a Broadcast Output feature for JackTrip that dramatically improved the time accuracy and audio quality of streaming from JackTrip sessions. But the main obstacle to JackTrip’s adoption by musicians, its complexity, remained untouched. That’s why in early 2021 Dan and I decided to start a new project focused on usability and a musician-friendly interface. That’s how FarPlay was born.

Initially we planned to utilize JackTrip code (which is open sourced under MIT license). But during this work I found that its architecture was hardly suited to our goals. It was designed as a UNIX command-line tool, not as a general purpose library. Thus I ended up writing a completely new implementation from scratch. I removed Jack dependency which allowed achieving even lower latency in some cases. Dan designed a GUI based on his experience using JackTrip in live-streaming and teaching others how to use it. We launched a closed pilot in August 2021 and an open beta in October 2021.