You’re muted — or are you?
April 16, 2022 1:44 PM Subscribe
Kassem Fawaz’s brother was on a videoconference with the microphone muted when he noticed that the microphone light was still on — indicating, inexplicably, that his microphone was being accessed. Alarmed, he asked Fawaz, an expert in online privacy and an assistant professor of electrical and computer engineering at the University of Wisconsin–Madison, to look into the issue. Fawaz and graduate student Yucheng Yang investigated whether this “mic-off-light-on” phenomenon was more widespread. They tried out many different videoconferencing applications on major operating systems, including iOS, Android, Windows and Mac, checking to see if the apps still accessed the microphone when it was muted.
Unfortunately, as this research remains unpublished, we’re unable to confirm the specific apps tested. So, for now, we can’t name and shame them.
However, the efficacy of this paper isn’t necessarily in doubt due to the fact that it’s been accepted to the 2022 Privacy Enhancing Technologies Symposium. We’ll just have to wait and see who gets name-dropped when the paper is published in June.
Abstract:
In the post-pandemic era, video conferencing apps (VCAs) have converted previously private spaces — bedrooms, living rooms, and kitchens — into semi-public extensions of the office. And for the most part, users have accepted these apps in their personal space, without much thought about the permission models that govern the use of their personal data during meetings. While access to a device’s video camera is carefully controlled, little has been done to ensure the same level of privacy for accessing the microphone. In this work, we ask the question: what happens to the microphone data when a user clicks the mute button in a VCA? We first conduct a user study to analyze users' understanding of the permission model of the mute button. Then, using runtime binary analysis tools, we trace raw audio in many popular VCAs as it traverses the app from the audio driver to the network. We find fragmented policies for dealing with microphone data among VCAs — some continuously monitor the microphone input during mute, and others do so periodically. One app transmits statistics of the audio to its telemetry servers while the app is muted. Using network traffic that we intercept en route to the telemetry server, we implement a proof-of-concept background activity classifier and demonstrate the feasibility of inferring the ongoing background activity during a meeting — cooking, cleaning, typing, etc. We achieved 81.9% macro accuracy on identifying six common background activities using intercepted outgoing telemetry packets when a user is muted.
Unfortunately, as this research remains unpublished, we’re unable to confirm the specific apps tested. So, for now, we can’t name and shame them.
However, the efficacy of this paper isn’t necessarily in doubt due to the fact that it’s been accepted to the 2022 Privacy Enhancing Technologies Symposium. We’ll just have to wait and see who gets name-dropped when the paper is published in June.
Abstract:
In the post-pandemic era, video conferencing apps (VCAs) have converted previously private spaces — bedrooms, living rooms, and kitchens — into semi-public extensions of the office. And for the most part, users have accepted these apps in their personal space, without much thought about the permission models that govern the use of their personal data during meetings. While access to a device’s video camera is carefully controlled, little has been done to ensure the same level of privacy for accessing the microphone. In this work, we ask the question: what happens to the microphone data when a user clicks the mute button in a VCA? We first conduct a user study to analyze users' understanding of the permission model of the mute button. Then, using runtime binary analysis tools, we trace raw audio in many popular VCAs as it traverses the app from the audio driver to the network. We find fragmented policies for dealing with microphone data among VCAs — some continuously monitor the microphone input during mute, and others do so periodically. One app transmits statistics of the audio to its telemetry servers while the app is muted. Using network traffic that we intercept en route to the telemetry server, we implement a proof-of-concept background activity classifier and demonstrate the feasibility of inferring the ongoing background activity during a meeting — cooking, cleaning, typing, etc. We achieved 81.9% macro accuracy on identifying six common background activities using intercepted outgoing telemetry packets when a user is muted.
I sure wish we had proper hardware off switches for surveillance equipment. It's relatively easy for a camera, you can put a sticker or shroud over it. But microphones are remarkably hard to disable if the hardware isn't designed with a real off switch.
posted by Nelson at 2:01 PM on April 16, 2022 [12 favorites]
posted by Nelson at 2:01 PM on April 16, 2022 [12 favorites]
what happens to the microphone data when a user clicks the mute button in a VCA?
as long as it isn't sent to the server and/or other users the software button is doing what it advertises.
I do use conference mics with hardware mute buttons for certain situations (like when I think a client may not be able to shut up). A gain knob can be helpful too. $20.
posted by snuffleupagus at 2:02 PM on April 16, 2022 [8 favorites]
as long as it isn't sent to the server and/or other users the software button is doing what it advertises.
I do use conference mics with hardware mute buttons for certain situations (like when I think a client may not be able to shut up). A gain knob can be helpful too. $20.
posted by snuffleupagus at 2:02 PM on April 16, 2022 [8 favorites]
I had a weird version of this happen on Teams a few weeks ago. I left a meeting, and could hear someone still talking. It sounded like, from their end, everyone had left the meeting. From my end, I had no access to the meeting chat, all my video lights were off, etc. But there was still an audio connection. Had to force close Teams to clear it (and rebooting the PC afterwards just to be safe).
posted by curious nu at 2:21 PM on April 16, 2022 [2 favorites]
posted by curious nu at 2:21 PM on April 16, 2022 [2 favorites]
Also if you're a security minded person, go update chrome, there's a new zero day.
Yes, another one.
posted by snuffleupagus at 2:26 PM on April 16, 2022 [7 favorites]
Yes, another one.
posted by snuffleupagus at 2:26 PM on April 16, 2022 [7 favorites]
Yeah, I guess I am not totally surprised about this. Or rather, the only bit that strikes me as particularly interesting is the part about being able to recover information about what the user is doing purely from the telemetry packets.
I am a bit disappointed that the authors declined to name the specific products. The whole academic norm of embargoing product details when security issues are found seems, overall, to be counterproductive and hostile to actual users. (I'm willing to hear arguments about embargoing details when completely unintentional vulnerabilities, i.e. buffer overflows or whatever, are found, so that the vendor can issue a patch, but I don't think this applies when the problem is due to intentional, designed-in behavior.) I'd much rather see researchers light those motherfuckers up in public. Name and shame. Show receipts. Users deserve to know what their software is doing behind their back.
It would probably be good if computers had a physical light—similar to the hardware-level lights on cameras—that lit up when a microphone was in use. However, I suspect it would probably be lit up pretty much all the time. As others have noted, having local software constantly monitor the mic is how Zoom et al can do the "You seem to be speaking but you're on mute, do you want to un-mute?" thing, which isn't terrible in and of itself.
I don't think that users necessarily expect that clicking the Mute button in Zoom will necessarily turn off their computer's microphone at a hardware level, or if they do, I'm not convinced that's a reasonable expectation. The expectation ought to be that the button causes nothing to be sent out to another party, not that it completely disables the mic.
But I think we are rapidly moving towards a world—in fact, we are there already, if you work in a security-conscious industry—where you simply cannot afford to trust that any device with a microphone and an internet uplink isn't listening to you. If you want to have a private conversation in a security-conscious environment, there are places for that: they're called SCIFs.
You do not bring cellphones in to a SCIF. You don't bring "dirty" computers into a SCIF (and you definitely don't take them out). You don't bring Apple Watches, Fitbits, iPads, Zunes, USB thumbdrives, and certainly not an Amazon Echo, into a SCIF. They can be as small as an 8x8 cargo container or as big as a whole office building. They are expensive, although mostly because the only ones who really care about Actual Security are organizations with a lot of money to burn.
But that's literally what it takes if you want to have a no-shit private conversation about something of interest to state-level actors in 2022. Which, by the way, is not just the inner workings of three-letter agencies. (With apologies to Pericles, just because you aren't interested in the PLA, doesn't mean the PLA isn't interested in you.)
We need to bring the cost of building SCIF-like spaces down to something that's accessible to regular people, or at least regular companies and organizations. In particular, we need to bring the cost of detection equipment that can tell you if something around you has a cellular modem built into it, down to reasonable levels. (I'm pretty confident you could build a semi-reliable one for a few dollars, and an actually reliable one for about $500, but I'd like to see that drop by an order of magnitude.)
None of this means that we shouldn't be naming-and-shaming companies and people for making software that does things a user wouldn't expect it to. That's gross and bad, and the people who do it should get their hands slapped (or cut off) fast and hard. But it is very likely a doomed mission. It's hard, probably impossible, to audit software as fast as it's created, and the average user probably isn't thinking about security when they install the latest cool app on their phone or computer. It'd be great if people did, and I personally do my best to try and get people to think about where their software comes from, but I'm aware it's a Quixotean task at best.
Unless you want to do your wanking in a SCIF (and it is discouraged, in my experience), society is eventually going to have to get over the fact that most people watch porn, masturbate, have sex, talk on the phone from the bathroom, smoke weed, use foul language, etc. The danger in these activities, at least from a security perspective, comes not from anything inherent in them, but from the ability of bad actors to use them for blackmail and leverage. The faster we realize that the danger is in allowing yourself to be blackmailed, rather than the admission you do the activity itself, the faster we remove the threat.
But for things that cannot ever be socially normalized and de-fanged that way, there's wire mesh and data diodes.
posted by Kadin2048 at 2:55 PM on April 16, 2022 [33 favorites]
I am a bit disappointed that the authors declined to name the specific products. The whole academic norm of embargoing product details when security issues are found seems, overall, to be counterproductive and hostile to actual users. (I'm willing to hear arguments about embargoing details when completely unintentional vulnerabilities, i.e. buffer overflows or whatever, are found, so that the vendor can issue a patch, but I don't think this applies when the problem is due to intentional, designed-in behavior.) I'd much rather see researchers light those motherfuckers up in public. Name and shame. Show receipts. Users deserve to know what their software is doing behind their back.
It would probably be good if computers had a physical light—similar to the hardware-level lights on cameras—that lit up when a microphone was in use. However, I suspect it would probably be lit up pretty much all the time. As others have noted, having local software constantly monitor the mic is how Zoom et al can do the "You seem to be speaking but you're on mute, do you want to un-mute?" thing, which isn't terrible in and of itself.
I don't think that users necessarily expect that clicking the Mute button in Zoom will necessarily turn off their computer's microphone at a hardware level, or if they do, I'm not convinced that's a reasonable expectation. The expectation ought to be that the button causes nothing to be sent out to another party, not that it completely disables the mic.
But I think we are rapidly moving towards a world—in fact, we are there already, if you work in a security-conscious industry—where you simply cannot afford to trust that any device with a microphone and an internet uplink isn't listening to you. If you want to have a private conversation in a security-conscious environment, there are places for that: they're called SCIFs.
You do not bring cellphones in to a SCIF. You don't bring "dirty" computers into a SCIF (and you definitely don't take them out). You don't bring Apple Watches, Fitbits, iPads, Zunes, USB thumbdrives, and certainly not an Amazon Echo, into a SCIF. They can be as small as an 8x8 cargo container or as big as a whole office building. They are expensive, although mostly because the only ones who really care about Actual Security are organizations with a lot of money to burn.
But that's literally what it takes if you want to have a no-shit private conversation about something of interest to state-level actors in 2022. Which, by the way, is not just the inner workings of three-letter agencies. (With apologies to Pericles, just because you aren't interested in the PLA, doesn't mean the PLA isn't interested in you.)
We need to bring the cost of building SCIF-like spaces down to something that's accessible to regular people, or at least regular companies and organizations. In particular, we need to bring the cost of detection equipment that can tell you if something around you has a cellular modem built into it, down to reasonable levels. (I'm pretty confident you could build a semi-reliable one for a few dollars, and an actually reliable one for about $500, but I'd like to see that drop by an order of magnitude.)
None of this means that we shouldn't be naming-and-shaming companies and people for making software that does things a user wouldn't expect it to. That's gross and bad, and the people who do it should get their hands slapped (or cut off) fast and hard. But it is very likely a doomed mission. It's hard, probably impossible, to audit software as fast as it's created, and the average user probably isn't thinking about security when they install the latest cool app on their phone or computer. It'd be great if people did, and I personally do my best to try and get people to think about where their software comes from, but I'm aware it's a Quixotean task at best.
Unless you want to do your wanking in a SCIF (and it is discouraged, in my experience), society is eventually going to have to get over the fact that most people watch porn, masturbate, have sex, talk on the phone from the bathroom, smoke weed, use foul language, etc. The danger in these activities, at least from a security perspective, comes not from anything inherent in them, but from the ability of bad actors to use them for blackmail and leverage. The faster we realize that the danger is in allowing yourself to be blackmailed, rather than the admission you do the activity itself, the faster we remove the threat.
But for things that cannot ever be socially normalized and de-fanged that way, there's wire mesh and data diodes.
posted by Kadin2048 at 2:55 PM on April 16, 2022 [33 favorites]
My USB mic has a mute button which turns off a light... and does not actually mute the microphone. It was pretty embarrassing when I figured this out after several meetings.
The only reliable way to mute it is yank the USB cable.
posted by Foosnark at 3:00 PM on April 16, 2022 [6 favorites]
The only reliable way to mute it is yank the USB cable.
posted by Foosnark at 3:00 PM on April 16, 2022 [6 favorites]
for a while I thought that it was kind of overblown when they mentioned at some recent-ish MacBook reveal or other that it has a design that physically disconnects the microphone from power when you close the lid, but maybe it turns out they were on to something
posted by DoctorFedora at 3:02 PM on April 16, 2022 [8 favorites]
posted by DoctorFedora at 3:02 PM on April 16, 2022 [8 favorites]
My USB mic has a mute button which turns off a light... and does not actually mute the microphone
Yikes! It makes sense that some of them could just be HID buttons, I guess. Meant to control the in-app mute function. If you get one with a volume knob, that's probably a real control (so you could just turn it all the way down) but it's always a good idea to do test calls with a new setup.
posted by snuffleupagus at 3:04 PM on April 16, 2022
Yikes! It makes sense that some of them could just be HID buttons, I guess. Meant to control the in-app mute function. If you get one with a volume knob, that's probably a real control (so you could just turn it all the way down) but it's always a good idea to do test calls with a new setup.
posted by snuffleupagus at 3:04 PM on April 16, 2022
Via Slashdot, The Register cites Cisco Webex in its report on this research.
posted by k3ninho at 3:05 PM on April 16, 2022 [1 favorite]
posted by k3ninho at 3:05 PM on April 16, 2022 [1 favorite]
some recent-ish MacBook reveal or other that it has a design that physically disconnects the microphone from power when you close the lid
By golly, this seems to be true on my 2019 Macbook Pro. Definitely a change from previous Macbooks, since I remember having a conference call using the speakers and mic from a closed Macbook (and external peripherals attached).
On hardware buttons - mygaming meeting headset has a physical mic on/off switch, but using it makes a loud noise to everyone else on the call, so I've avoided it. I can unplug the mic entirely from the headset which probably works but if I were ultra-paranoid I would use the cable that is physically unable to transmit microphone noise.
posted by meowzilla at 4:12 PM on April 16, 2022
By golly, this seems to be true on my 2019 Macbook Pro. Definitely a change from previous Macbooks, since I remember having a conference call using the speakers and mic from a closed Macbook (and external peripherals attached).
On hardware buttons - my
posted by meowzilla at 4:12 PM on April 16, 2022
Here's Apple's page on it: Hardware microphone disconnect
Also applies to iPads which are in a case that has the nifty "close the cover to turn off the screen" feature - the mic is shut off when the iPad is "closed".
posted by meowzilla at 4:15 PM on April 16, 2022 [3 favorites]
Also applies to iPads which are in a case that has the nifty "close the cover to turn off the screen" feature - the mic is shut off when the iPad is "closed".
posted by meowzilla at 4:15 PM on April 16, 2022 [3 favorites]
For Zoom or Teams if it's not just me in the room (so there are multiple mics and/or cameras), or if I'm giving some kind of presentation, I'm usually feeding everything through OBS anyway. (Which lets me do stuff like superimpose my webcam in a corner over a document cam.) Or some other software mixer that shows me levels. And the conference app is seeing its output as its inputs.
I usually do have at least two screens running so I can see the conference and the controls. Sometimes three if there's content I need to access or present (or I may use a tablet and/or separate laptop for that.)
And of course there's always the option to use a standalone mic and a mic mixer with the built-in input, or a USB audio interface that has physical controls.
posted by snuffleupagus at 4:22 PM on April 16, 2022 [1 favorite]
I usually do have at least two screens running so I can see the conference and the controls. Sometimes three if there's content I need to access or present (or I may use a tablet and/or separate laptop for that.)
And of course there's always the option to use a standalone mic and a mic mixer with the built-in input, or a USB audio interface that has physical controls.
posted by snuffleupagus at 4:22 PM on April 16, 2022 [1 favorite]
I didn’t find a link in a few minutes of searching, but previously on Metafilter we’ve discussed how the always-on analysis which enables “Hey Siri” and the like is also passed to advertisers, which is why some people see ads for swim goggles when the people next to them start talking about the beach.
posted by fantabulous timewaster at 5:44 PM on April 16, 2022 [1 favorite]
posted by fantabulous timewaster at 5:44 PM on April 16, 2022 [1 favorite]
The only reliable way to mute it is yank the USB cable.
I think it's likely that a given application will just switch to using the computer's built-in microphone, so, maybe not.
posted by amtho at 6:02 PM on April 16, 2022 [5 favorites]
I think it's likely that a given application will just switch to using the computer's built-in microphone, so, maybe not.
posted by amtho at 6:02 PM on April 16, 2022 [5 favorites]
previously on Metafilter we’ve discussed
maybe these threads, about Echo & Alexa?
Here's Looking at You
Surveillance has never been more affordable
posted by snuffleupagus at 6:05 PM on April 16, 2022
maybe these threads, about Echo & Alexa?
Here's Looking at You
Surveillance has never been more affordable
posted by snuffleupagus at 6:05 PM on April 16, 2022
I think it's likely that a given application will just switch to using the computer's built-in microphone, so, maybe not.
I've got a desktop with parts I specced myself, so unless there's a secret hidden mic in the motherboard or SSD I'm pretty sure I just have the external mic.
posted by Foosnark at 6:15 PM on April 16, 2022
I've got a desktop with parts I specced myself, so unless there's a secret hidden mic in the motherboard or SSD I'm pretty sure I just have the external mic.
posted by Foosnark at 6:15 PM on April 16, 2022
Awesome! As long as other people with less custom setups don't think they can be confident the same way, it's great to have that in your back pocket!
posted by amtho at 6:30 PM on April 16, 2022
posted by amtho at 6:30 PM on April 16, 2022
Note that previous threads expressly did not conclude that anybody was using hot word detection to sell ads, quite the contrary.
This sure seems like a tempest in a teacup.
posted by abulafa at 6:31 PM on April 16, 2022 [13 favorites]
This sure seems like a tempest in a teacup.
posted by abulafa at 6:31 PM on April 16, 2022 [13 favorites]
Weird. The Register article doesn't mention Webex using this to talk to the Cisco video-conferencing boards in meeting rooms. Lets you initiate a call on your laptop and then transfer it to the VC system. I have no doubt the mics listen in to all manner of stuff but there is often another explanation for it. Probably a service that could be stopped or disabled to prevent it. Doesn't account for non Cisco stuff tho'.
posted by phigmov at 6:33 PM on April 16, 2022
posted by phigmov at 6:33 PM on April 16, 2022
Also if you're a security minded person, go update chrome, there's a new zero day.
Or use Brave, which the Chrome browser with more privacy and security (caveat: it blocks ads and tracking great but has it's own semi-spammy crap you can turn off).
posted by kirkaracha at 6:50 PM on April 16, 2022 [1 favorite]
Or use Brave, which the Chrome browser with more privacy and security (caveat: it blocks ads and tracking great but has it's own semi-spammy crap you can turn off).
posted by kirkaracha at 6:50 PM on April 16, 2022 [1 favorite]
Framework laptops as well as being repairable and modular have Separate switches for physically disabling webcam/microphone
posted by lalochezia at 7:01 PM on April 16, 2022 [4 favorites]
posted by lalochezia at 7:01 PM on April 16, 2022 [4 favorites]
This exploit hit Brave as well. It has privacy features (which I won't debate the merits of), it shouldn't be considered more secure. It's a Chromium fork.
posted by snuffleupagus at 7:12 PM on April 16, 2022 [4 favorites]
posted by snuffleupagus at 7:12 PM on April 16, 2022 [4 favorites]
Most (business, anyway) laptops are repairable, but few have hardware controls for the microphone. Some do have camera shutters and some have a reasonably easy, though not easily accessible, means of disconnecting the internal camera and microphone(s), though.
I'm not at all surprised that programs are still accessing the microphone even when muted. It's pretty common for them to just throw away the data rather than transmitting it. It does surprise me that they continue to process the audio and even submit statistics to the server.
posted by wierdo at 7:13 PM on April 16, 2022
I'm not at all surprised that programs are still accessing the microphone even when muted. It's pretty common for them to just throw away the data rather than transmitting it. It does surprise me that they continue to process the audio and even submit statistics to the server.
posted by wierdo at 7:13 PM on April 16, 2022
Can you just turn your sound down? I only zoom on my phone, I don't have a microphone or camera in my desktop. But I do have a CO2 monitor with some sort of proximity sensor, and a little fashing green light when I get out of bed. I think I will start hanging a garment on it.
posted by Oyéah at 7:40 PM on April 16, 2022
posted by Oyéah at 7:40 PM on April 16, 2022
The whole academic norm of embargoing product details when security issues are found seems, overall, to be counterproductive and hostile to actual users.
Could they be following the security industry norm of timed disclosure? It's a good compromise between making sure the general public is informed about security failures and minimising the harm from these failures. The standard time limit gives companies an incentive to not fight researchers and also typically gives them enough time to patch the issue before it's widely disclosed.
And sometimes you still get people named and shamed! Like Zoom. Zoom is dodgy as hell.
posted by Merus at 7:53 PM on April 16, 2022
Could they be following the security industry norm of timed disclosure? It's a good compromise between making sure the general public is informed about security failures and minimising the harm from these failures. The standard time limit gives companies an incentive to not fight researchers and also typically gives them enough time to patch the issue before it's widely disclosed.
And sometimes you still get people named and shamed! Like Zoom. Zoom is dodgy as hell.
posted by Merus at 7:53 PM on April 16, 2022
previously on Metafilter we’ve discussed how the always-on analysis which enables “Hey Siri” and the like is also passed to advertisers
Whoa. That is not known to be the case, not at all, and it would be a big deal if it were true.
The "hey Siri" or "hey Alexa" features work by virtue of a very limited onboard speech-recognition capability on the device itself, which isn't enough computational horsepower to do actual speaker-independent voice recognition, but is just enough to recognize the "wake words" as spoken by most people. (This is why, incidentally, you can't just rename your "Alexa" to "Jeff Bezos You Motherfucker"—the built-in speech recognizer is pretty finely tuned for one of a very small set of names/phrases. This might change with better AI acceleration hardware, though.)
What most devices do is constantly listen, digitize/encode, and store the last few seconds of audio in a buffer. When the built-in SR thinks it heard you say the "wake word", it sends the contents of the buffer plus the ensuing live audio stream to a server in a datacenter somewhere. The datacenter server—much more powerful than the $30 device on your desk—confirms that you indeed did say the wakeword, and then parses the rest of the audio for valid commands.
There have been cases where Alexa/Siri-type systems have sent unintended audio to servers, generally because they aren't 100% reliable about hearing the wakewords and almost certainly are tuned to false-positive more than they false-negative. And those false-positives (the local device thought you said "hey Siri", but on further review the server software decides you didn't) were in some cases being stored to improve the algorithms, or sent to underpaidmeatbots humans to manually classify. (I think Apple now has an explicit opt-in for this, not sure about Amazon or Google.)
You can pretty trivially ensure that an Alexa-type device isn't constantly sending an audio stream to its owner by looking at the network traffic it generates. I've done this. There's not nearly enough traffic coming from an Amazon Echo when it's not in use to contain a constant audio stream, even with good compression. (They are real fuckin' chatty, though, by computer standards.)
The "haha my Echo heard me talking about carrots so now I only see ads for carrots!" thing is... not really how those systems work. It's how people imagine those systems work, because the reality is honestly much more complicated, and potentially more troubling. And there's also a lot of confirmation bias involved. (We notice when Amazon recommends something to us that we just talked about, but ignore the many "misses" when it recommends something irrelevant. So the systems as a whole seem 'smarter' than they are.)
Amazon/Google/Facebook don't need to listen to a constant audio feed from your Echo-equivalent device, like they're the Stasi in 1960s East Germany.
They already know, to a fair degree of certainty, who you are, who your friends are, who your relatives are, and when they're in your house. They have a good idea of what websites you're visiting and what you might have searched for. They know what ads you've seen, and have metrics on how long you engaged with them. So when your friend mentions that they just read this cool article about the all-carrot diet and suddenly you start getting Instagram ads for carrot powder, it's not because someone was listening to your conversation, but more like some statistical model somewhere decided that you were slightly more likely to buy carrot powder than the average person, since there's a good chance you spent more than an hour in the presence of a close friend who just had greater-than average engagement with 8 carrot-diet-related ads in the last 6 days (and whose roommate just bought 6 liters of carrot juice last Tuesday with their coupon card, etc. etc.). The end result: carrot ads, carrot ads everywhere.
This is fairly important for people to understand, because a lack of understanding is likely to lead to totally nonsense, toothless regulation. The surveillance-capitalism system is far more complex and far more pervasive than the average person assumes it is.
posted by Kadin2048 at 8:06 PM on April 16, 2022 [101 favorites]
Whoa. That is not known to be the case, not at all, and it would be a big deal if it were true.
The "hey Siri" or "hey Alexa" features work by virtue of a very limited onboard speech-recognition capability on the device itself, which isn't enough computational horsepower to do actual speaker-independent voice recognition, but is just enough to recognize the "wake words" as spoken by most people. (This is why, incidentally, you can't just rename your "Alexa" to "Jeff Bezos You Motherfucker"—the built-in speech recognizer is pretty finely tuned for one of a very small set of names/phrases. This might change with better AI acceleration hardware, though.)
What most devices do is constantly listen, digitize/encode, and store the last few seconds of audio in a buffer. When the built-in SR thinks it heard you say the "wake word", it sends the contents of the buffer plus the ensuing live audio stream to a server in a datacenter somewhere. The datacenter server—much more powerful than the $30 device on your desk—confirms that you indeed did say the wakeword, and then parses the rest of the audio for valid commands.
There have been cases where Alexa/Siri-type systems have sent unintended audio to servers, generally because they aren't 100% reliable about hearing the wakewords and almost certainly are tuned to false-positive more than they false-negative. And those false-positives (the local device thought you said "hey Siri", but on further review the server software decides you didn't) were in some cases being stored to improve the algorithms, or sent to underpaid
You can pretty trivially ensure that an Alexa-type device isn't constantly sending an audio stream to its owner by looking at the network traffic it generates. I've done this. There's not nearly enough traffic coming from an Amazon Echo when it's not in use to contain a constant audio stream, even with good compression. (They are real fuckin' chatty, though, by computer standards.)
The "haha my Echo heard me talking about carrots so now I only see ads for carrots!" thing is... not really how those systems work. It's how people imagine those systems work, because the reality is honestly much more complicated, and potentially more troubling. And there's also a lot of confirmation bias involved. (We notice when Amazon recommends something to us that we just talked about, but ignore the many "misses" when it recommends something irrelevant. So the systems as a whole seem 'smarter' than they are.)
Amazon/Google/Facebook don't need to listen to a constant audio feed from your Echo-equivalent device, like they're the Stasi in 1960s East Germany.
They already know, to a fair degree of certainty, who you are, who your friends are, who your relatives are, and when they're in your house. They have a good idea of what websites you're visiting and what you might have searched for. They know what ads you've seen, and have metrics on how long you engaged with them. So when your friend mentions that they just read this cool article about the all-carrot diet and suddenly you start getting Instagram ads for carrot powder, it's not because someone was listening to your conversation, but more like some statistical model somewhere decided that you were slightly more likely to buy carrot powder than the average person, since there's a good chance you spent more than an hour in the presence of a close friend who just had greater-than average engagement with 8 carrot-diet-related ads in the last 6 days (and whose roommate just bought 6 liters of carrot juice last Tuesday with their coupon card, etc. etc.). The end result: carrot ads, carrot ads everywhere.
This is fairly important for people to understand, because a lack of understanding is likely to lead to totally nonsense, toothless regulation. The surveillance-capitalism system is far more complex and far more pervasive than the average person assumes it is.
posted by Kadin2048 at 8:06 PM on April 16, 2022 [101 favorites]
Advertising by contagion. Au courant.
posted by snuffleupagus at 8:21 PM on April 16, 2022
posted by snuffleupagus at 8:21 PM on April 16, 2022
I've got a desktop with parts I specced myself, so unless there's a secret hidden mic in the motherboard or SSD I'm pretty sure I just have the external mic.
There are a surprising number of electronic parts that can be made to act like a microphone in a pinch, if you have full control of the hardware. Mechanical hard drives, for instance. Speakers (even small, crappy ones). I would be very surprised if someone clever couldn't use a touchscreen as one, too. Probably a bunch more ways, if you had some money and a few motivated postdocs.
I would not assume that having built a computer from parts really adds that much in the way of security. In some cases, depending on your threat model, it might actually add a number of additional opportunities for supply chain tampering.
But again, if you think you might be of interest to nation-state actors, you need to take security precautions like a nation-state actor, which means not having conversations outside of shielded rooms that never contain 'dirty' electronics, and only doing data processing on air-gapped equipment inside those rooms which is never removed except for a one-way trip to an industrial incinerator. (And TBH, lots of nation-state actors aren't good about this stuff and leak information like goddamn sieves, because people are fundamentally bad at security.)
posted by Kadin2048 at 8:23 PM on April 16, 2022 [3 favorites]
There are a surprising number of electronic parts that can be made to act like a microphone in a pinch, if you have full control of the hardware. Mechanical hard drives, for instance. Speakers (even small, crappy ones). I would be very surprised if someone clever couldn't use a touchscreen as one, too. Probably a bunch more ways, if you had some money and a few motivated postdocs.
I would not assume that having built a computer from parts really adds that much in the way of security. In some cases, depending on your threat model, it might actually add a number of additional opportunities for supply chain tampering.
But again, if you think you might be of interest to nation-state actors, you need to take security precautions like a nation-state actor, which means not having conversations outside of shielded rooms that never contain 'dirty' electronics, and only doing data processing on air-gapped equipment inside those rooms which is never removed except for a one-way trip to an industrial incinerator. (And TBH, lots of nation-state actors aren't good about this stuff and leak information like goddamn sieves, because people are fundamentally bad at security.)
posted by Kadin2048 at 8:23 PM on April 16, 2022 [3 favorites]
if you think you might be of interest to nation-state actors
That's a very, very small subset of users compared to the victims of surveillance state capitalism like all of us here. It's a really interesting theoretical topic, but the practical world is different.
a very limited onboard speech-recognition capability on the device itself
Of relevance to that: Is Google Spying on your Conversations?. An interesting and obscure technical blog post from someone who worked on the infrastructure for "OK Google" voice prompts.
That's a very, very small subset of users compared to the victims of surveillance state capitalism like all of us here. It's a really interesting theoretical topic, but the practical world is different.
a very limited onboard speech-recognition capability on the device itself
Of relevance to that: Is Google Spying on your Conversations?. An interesting and obscure technical blog post from someone who worked on the infrastructure for "OK Google" voice prompts.
So, I would have been directly involved in any code that did the kind of conversational spying that many people incorrectly suspect is happening, and I’m in a good position to categorically say it isn’t. Why should you trust me though?posted by Nelson at 8:48 PM on April 16, 2022 [2 favorites]
The only way to be sure you're mute is to cut the power or unplug. Many USB hubs have power switches for individual ports. Use it.
The only way to be sure you're not seen is the block the camera (like a physical shutter on the webcam).
posted by kschang at 8:57 PM on April 16, 2022
The only way to be sure you're not seen is the block the camera (like a physical shutter on the webcam).
posted by kschang at 8:57 PM on April 16, 2022
This is why, incidentally, you can't just rename your "Alexa" to "Jeff Bezos You Motherfucker"
At least some of the hotword detection chips Google has been using since the early days have no problem detecting "Hey Fuckface!" I used it that way for a while. That was back when it was intended only to detect "OK Google".
Regardless, the ability to push (good) natural language processing into end devices is rapidly improving. It's not yet to the point where it can be done in super cheap Internet of Shit devices, but it's frankly shocking how good the latest Pixels are now that they can run the ML model locally on Tensor. It seems more accurate than the cloud processing (but probably isn't) and is lightning fast.
In the not too distant future it's going to be a lot harder to verify whether or not something.. unexpected..is happening behind the scenes. I'm not too worried about Google themselves because their incentives aren't oriented toward sharing the actual data. They'd much rather sell better ads. That is where they make the vast majority of their money, after all. However, there are some serious implications about what they could do quietly given legal process.
posted by wierdo at 9:47 PM on April 16, 2022 [2 favorites]
At least some of the hotword detection chips Google has been using since the early days have no problem detecting "Hey Fuckface!" I used it that way for a while. That was back when it was intended only to detect "OK Google".
Regardless, the ability to push (good) natural language processing into end devices is rapidly improving. It's not yet to the point where it can be done in super cheap Internet of Shit devices, but it's frankly shocking how good the latest Pixels are now that they can run the ML model locally on Tensor. It seems more accurate than the cloud processing (but probably isn't) and is lightning fast.
In the not too distant future it's going to be a lot harder to verify whether or not something.. unexpected..is happening behind the scenes. I'm not too worried about Google themselves because their incentives aren't oriented toward sharing the actual data. They'd much rather sell better ads. That is where they make the vast majority of their money, after all. However, there are some serious implications about what they could do quietly given legal process.
posted by wierdo at 9:47 PM on April 16, 2022 [2 favorites]
WebEx chat client user here (software formally known as WebEx Teams).
Check your WebEx settings for a feature called "ultrasound". It basically uses your computer microphone all the time while the feature is enabled to be and to detect and automatically connect to Cisco teleconference devices nearby.
And the feature is enabled by default. I never realized this until apple started putting the mic-in-use notifier dot into the to right corner of the screen.
posted by neuracnu at 1:11 AM on April 17, 2022 [7 favorites]
Check your WebEx settings for a feature called "ultrasound". It basically uses your computer microphone all the time while the feature is enabled to be and to detect and automatically connect to Cisco teleconference devices nearby.
And the feature is enabled by default. I never realized this until apple started putting the mic-in-use notifier dot into the to right corner of the screen.
posted by neuracnu at 1:11 AM on April 17, 2022 [7 favorites]
And realize that your hardware may have unlisted features. Last time I bought a TV all I was looking for was a display for my cable box, and it irritated me a lot that during setup it insisted on being connected to the internet (either via ethernet cable or wifi) and I gave it a 30 seconds on a phone hotspot to get past that bit of setup and it has never seen a network connection since. And yet that was not enough. A few weeks later I was writing a bluetooth scanner for a project and the first run produced a list of nearby bluetooth devices. I was able to recognize most of them (by type, as I was most likely seeing a few of my neighbors devices as well) and saw one that, by the description record it broadcast, must have been that same TV.
So I checked the TV manual: it did not list any bluetooth features. I checked the manufacturers site: it did not list any bluetooth features. After a while I did find out that the bluetooth broadcast was used by the "universal remote" app published by the manufacturer to figure out the type and features available. Which was BS since that could have been done on the first connection over wifi, but whoever designed it figured out "hey, who cares, that chip we put on the board for wifi also supports bluetooth so why not".
So my TV had an undocumented broadcasting feature. I am sure it's nice for any potential burglar to be able to walk along a row of houses with a bluetooth scanner and immediately know where the biggest screens can be stolen from, without the owners being able to do anything about it.
posted by DreamerFi at 5:56 AM on April 17, 2022 [6 favorites]
So I checked the TV manual: it did not list any bluetooth features. I checked the manufacturers site: it did not list any bluetooth features. After a while I did find out that the bluetooth broadcast was used by the "universal remote" app published by the manufacturer to figure out the type and features available. Which was BS since that could have been done on the first connection over wifi, but whoever designed it figured out "hey, who cares, that chip we put on the board for wifi also supports bluetooth so why not".
So my TV had an undocumented broadcasting feature. I am sure it's nice for any potential burglar to be able to walk along a row of houses with a bluetooth scanner and immediately know where the biggest screens can be stolen from, without the owners being able to do anything about it.
posted by DreamerFi at 5:56 AM on April 17, 2022 [6 favorites]
Amazon/Google/Facebook don't need to listen to a constant audio feed from your Echo-equivalent device, like they're the Stasi in 1960s East Germany.
Still doesn’t mean I’ll ever allow an Alexa-enabled device into my home. I know I’m being targeted for ads, but that doesn’t mean I have to help them do it.
Re the FPP, I’m happy to hear that WebEx is the culprit here. Of all the videoconferencing apps we’ve used at work over the past two years, WebEx has consistently been the worst. Worst interface, most frustrating to use, all that. I’d asked my boss if we could stop using it and switch to another platform well before this article came out, and she was also fed up with it, this just gives us one more reason to switch.
posted by caution live frogs at 7:07 AM on April 17, 2022
Still doesn’t mean I’ll ever allow an Alexa-enabled device into my home. I know I’m being targeted for ads, but that doesn’t mean I have to help them do it.
Re the FPP, I’m happy to hear that WebEx is the culprit here. Of all the videoconferencing apps we’ve used at work over the past two years, WebEx has consistently been the worst. Worst interface, most frustrating to use, all that. I’d asked my boss if we could stop using it and switch to another platform well before this article came out, and she was also fed up with it, this just gives us one more reason to switch.
posted by caution live frogs at 7:07 AM on April 17, 2022
TVs these days also have microphones. And given the very shady and shitty history of smart TVs spying on you without disclosure, it seems likely that at least some of them are using the audio stream for profiling and advertising.
posted by Nelson at 7:40 AM on April 17, 2022 [1 favorite]
posted by Nelson at 7:40 AM on April 17, 2022 [1 favorite]
I very much want one-button mic mute and camera power at the hardware (ideally) or OS level. Leaving it to applications is asking for trouble.
posted by Nothing at 9:31 AM on April 17, 2022
posted by Nothing at 9:31 AM on April 17, 2022
Microphones? TVs have screen capture software running that can upload live footage or snapshots of everything shown on the TV from any input. I know this because I use RootMyTv to repurpose that ability to drive LED ambient lighting.
(The tv gets network access but is firewalled off from the internet)
posted by Nonsteroidal Anti-Inflammatory Drug at 11:33 AM on April 17, 2022 [1 favorite]
(The tv gets network access but is firewalled off from the internet)
posted by Nonsteroidal Anti-Inflammatory Drug at 11:33 AM on April 17, 2022 [1 favorite]
Could they be following the security industry norm of timed disclosure?
no. if they are, it's a misapplication of the principle. this isn't a bug or exploit they discovered, it's a planned and engineered secret feature. both intentionally hidden from users, and purposefully deceitful through the UI design (mute does not mean what you think it means).
posted by j_curiouser at 12:59 PM on April 17, 2022 [2 favorites]
no. if they are, it's a misapplication of the principle. this isn't a bug or exploit they discovered, it's a planned and engineered secret feature. both intentionally hidden from users, and purposefully deceitful through the UI design (mute does not mean what you think it means).
posted by j_curiouser at 12:59 PM on April 17, 2022 [2 favorites]
j_curiouser makes a good point: there is a huge difference between a bug and an engineered exploitation of the user. A big problem that I don’t feel is widely recognized is that modern devices and software are so bloody complicated that without a big engineering team you have no idea what is really going on.
So there’s a “mute” “button” on the screen, and you press it and … a light goes on. You can do a few experiments and maybe convince yourself that the people on the other end can’t hear you (with all the preferences and things as they’re set out of the box).
Where does the voice information get cut off? Is it within the perimeter of your computer? Does it get to the server that relays traffic between you and your fellow conferees? Does it get onto their computers but not sent to their speakers by the conference software? Those three possibilities (and there are a lot more than three) have very different security implications.
All you can really know is that the company producing the software has little incentive to do it right, and there are actually a bunch of hardware and software companies and independent programmers and so forth with no overarching plan to ensure anything … except that most of them are interested in getting paid as much as they can. Which shouldn’t give you a warm fuzzy.
posted by Gilgamesh's Chauffeur at 5:39 AM on April 18, 2022 [1 favorite]
So there’s a “mute” “button” on the screen, and you press it and … a light goes on. You can do a few experiments and maybe convince yourself that the people on the other end can’t hear you (with all the preferences and things as they’re set out of the box).
Where does the voice information get cut off? Is it within the perimeter of your computer? Does it get to the server that relays traffic between you and your fellow conferees? Does it get onto their computers but not sent to their speakers by the conference software? Those three possibilities (and there are a lot more than three) have very different security implications.
All you can really know is that the company producing the software has little incentive to do it right, and there are actually a bunch of hardware and software companies and independent programmers and so forth with no overarching plan to ensure anything … except that most of them are interested in getting paid as much as they can. Which shouldn’t give you a warm fuzzy.
posted by Gilgamesh's Chauffeur at 5:39 AM on April 18, 2022 [1 favorite]
Re-mentioning that the Framework laptops have hardware switches for both microphone and camera. Activated both last night, and saw them detected in DMESG as if they were just plugged in.
posted by Orb2069 at 6:56 AM on April 18, 2022 [2 favorites]
posted by Orb2069 at 6:56 AM on April 18, 2022 [2 favorites]
Still doesn’t mean I’ll ever allow an Alexa-enabled device into my home. I know I’m being targeted for ads, but that doesn’t mean I have to help them do it.
I have no horse in this race, I don't care either way—my point was, though, if what you care about is "not providing ad-targeting data", having an Alexa or not having an Alexa in your house really isn't making that much of a difference.
Because, again, they are not listening to a livestream of audio from it at Amazon HQ. You can trivially prove this with the most basic of networking sniffing tools. All you need to do is watch the packet stream coming from an Echo device when it's quiescent (and, again, they're fuckin' chatty little bastards, but I suspect most are just keep-alive and telemetry packets, and the occasional software update and the like), and then say "Hey Alexa tell me the weather" (or whatever) and watch the packet volume suddenly jump as the audio gets sent to AWS. There just isn't enough bandwidth being used to contain a constant audio stream, and it's really obvious when audio is being sent.
I mean, personally I'm not feeding any information into my Echo that's particularly useful for ad targeting. (I guess it tells Amazon that I'm pretty interested in the weather. With some analysis maybe they can back out some useful insights about my upcoming umbrella purchase habits, I suppose. But they've never bothered to try and sell me one.)
If you're interested in breaking Amazon (and many other companies') supply of data that's used for ad targeting, there are things you can do. But they are harder than just not buying an Echo.
The biggest thing is probably to stop using any store discount cards (CVS ExtraCare, Giant Rewards, etc.). Doesn't matter that you entered a fake name, doesn't matter that you picked it up off the ground in the parking lot. By tying the purchase to the payment method and tying various purchases together, they can learn a lot about you. Definitely don't use any pharmacy discount cards (IIRC GoodRx has an alarmingly broad user agreement). No coupons, unless you're sure they're mass-printed (and not custom printed for you). Avoid anything with a QR or other 2-D barcode that can hold more than a few bits of information.
Honestly, you probably don't want to use credit cards at all, since they're almost all certainly selling your data onto brokers who aggregate it and pass it along to other companies, who do their own aggregation and analysis. (Amexco seems to be the best, but by how much isn't clear.)
You'll want to disable all location services on your phone, since huge amounts of consumer data can be backed out of geolocation records (what stores you visited, how long, with whom, etc., all of which can be combined with financial transaction data to de-anonymize it). Of course, your cell phone company has your location data and may just be selling it anyway, regardless of what settings you use on your phone.
It wouldn't surprise me, as more electric, water, and gas utilities roll out "smart meter" technology with high-resolution data loggers, that they'll start selling their data on to brokers, too. (So they can tell when you get home, what appliances you use, whether you're cooking or ordering in, how often you shower, etc.) Good luck opting out of that.
The privacy impact of an Echo-like device is a lot less than I think most people realize, and the implications of seemingly innocuous actions that we do unthinkingly—like carrying a phone into a store, paying with a credit card, or going somewhere with a friend—are much higher than they appear, or were a few years ago.
posted by Kadin2048 at 10:40 PM on April 18, 2022 [6 favorites]
I have no horse in this race, I don't care either way—my point was, though, if what you care about is "not providing ad-targeting data", having an Alexa or not having an Alexa in your house really isn't making that much of a difference.
Because, again, they are not listening to a livestream of audio from it at Amazon HQ. You can trivially prove this with the most basic of networking sniffing tools. All you need to do is watch the packet stream coming from an Echo device when it's quiescent (and, again, they're fuckin' chatty little bastards, but I suspect most are just keep-alive and telemetry packets, and the occasional software update and the like), and then say "Hey Alexa tell me the weather" (or whatever) and watch the packet volume suddenly jump as the audio gets sent to AWS. There just isn't enough bandwidth being used to contain a constant audio stream, and it's really obvious when audio is being sent.
I mean, personally I'm not feeding any information into my Echo that's particularly useful for ad targeting. (I guess it tells Amazon that I'm pretty interested in the weather. With some analysis maybe they can back out some useful insights about my upcoming umbrella purchase habits, I suppose. But they've never bothered to try and sell me one.)
If you're interested in breaking Amazon (and many other companies') supply of data that's used for ad targeting, there are things you can do. But they are harder than just not buying an Echo.
The biggest thing is probably to stop using any store discount cards (CVS ExtraCare, Giant Rewards, etc.). Doesn't matter that you entered a fake name, doesn't matter that you picked it up off the ground in the parking lot. By tying the purchase to the payment method and tying various purchases together, they can learn a lot about you. Definitely don't use any pharmacy discount cards (IIRC GoodRx has an alarmingly broad user agreement). No coupons, unless you're sure they're mass-printed (and not custom printed for you). Avoid anything with a QR or other 2-D barcode that can hold more than a few bits of information.
Honestly, you probably don't want to use credit cards at all, since they're almost all certainly selling your data onto brokers who aggregate it and pass it along to other companies, who do their own aggregation and analysis. (Amexco seems to be the best, but by how much isn't clear.)
You'll want to disable all location services on your phone, since huge amounts of consumer data can be backed out of geolocation records (what stores you visited, how long, with whom, etc., all of which can be combined with financial transaction data to de-anonymize it). Of course, your cell phone company has your location data and may just be selling it anyway, regardless of what settings you use on your phone.
It wouldn't surprise me, as more electric, water, and gas utilities roll out "smart meter" technology with high-resolution data loggers, that they'll start selling their data on to brokers, too. (So they can tell when you get home, what appliances you use, whether you're cooking or ordering in, how often you shower, etc.) Good luck opting out of that.
The privacy impact of an Echo-like device is a lot less than I think most people realize, and the implications of seemingly innocuous actions that we do unthinkingly—like carrying a phone into a store, paying with a credit card, or going somewhere with a friend—are much higher than they appear, or were a few years ago.
posted by Kadin2048 at 10:40 PM on April 18, 2022 [6 favorites]
So best practice would be to mute at the OS level rather than individual app? Pretty sure windows can do that fairly easily, I know Linux can on virtually every sound setup I'm familiar with, don't know about mac or ios, certainly not easy if it's even possible in android.
Obviously hardware level is better still, but for all the laptops and tablets with built in mics out there, just going to your sound icon or mic preferences or whatever to mute could be a solution for now?
posted by Dysk at 5:51 AM on April 19, 2022
Obviously hardware level is better still, but for all the laptops and tablets with built in mics out there, just going to your sound icon or mic preferences or whatever to mute could be a solution for now?
posted by Dysk at 5:51 AM on April 19, 2022
« Older A drab, gray dream, ‘The Secrets of Dumbledore’ is... | Music From Nancy (1979) Newer »
This thread has been archived and is closed to new comments
posted by emelenjr at 1:52 PM on April 16, 2022 [50 favorites]