Facebook - It Can't Know Enough About Us

Updated September 2019

With the recent news that Facebook has allowed other big tech companies to access its users personal data under its "integration partnerships" as shown here:

...users have very good reason to be concerned about another recent development in "Facebook's World" that receive very little traction in the mainstream media which spends its energies worrying about Facebook's share price and its impact on the 2016 American election.

On June 14, 2018, Patent Number US20180167677 otherwise known as "Broadcast content view analysis based on ambient audio recording" was published as shown here:

Here is a rather technical summary of the technology involved as quoted from the patent application:

"Described methods, systems and computer program readable storage medium storing executable computer program instructions provide solutions for analyzing view of broadcast content for each user of an online system in a household based on recording of ambient audio of the broadcast content. Ambient audio/sounds refer to distinct and subtle sounds of a particular location created by the environment of the location, such as machinery noise, the sound of distant human movement and speech, creaks from thermal contraction, and air conditioning and plumbing noises in a household. A particular ambient audio/sound can be identified by an ambient audio fingerprint or signature (e.g., a binary code) generated from a subset of the ambient sound, e.g., an audio feature. An example of an audio feature associated with a broadcast sponsored content item includes high frequency modulated voice closer to 20 kHz, which is a non-human hearable digital sound, but machine recognizable set of Morse-style sounds.

     One embodiment includes a method for analyzing broadcast content view by individuals in a household based on ambient audio recording. In the embodiment, the household has a broadcasting device that displays broadcast content to the individuals. Each of the individuals is a user of an online system and is associated with a client device where a software application module is executed. The software application module detects one or more broadcasting signals of a content item broadcasted to a household by the broadcasting device. In one embodiment, detecting the broadcasting signals comprises detecting whether the client device is in the vicinity of the broadcasting device. It is noted that when the client device is in the vicinity of the broadcasting device, the individual associated with the client device is very likely to be viewing a content item being displayed on the broadcasting device. Upon detecting the broadcasting signals, the client device records the ambient audio of the content item. The software application module further extracts an audio feature from the recorded ambient audio to generate identifying information about the content item, e.g., an ambient audio fingerprint. The software application module sends a data set, which comprises the ambient audio fingerprint, time information of the recorded ambient audio, and an identifier of the individual, to the online system.

     The online system, for each data set received from the software application module, identifies the corresponding individual and content item, and determines whether there was an impression of the identified content item by the identified individual, i.e., whether the identified individual viewed the identified content item. Responsive to determination that there was an impression of the identified content item on the identified individual, the online system logs impression associated with the identified individual and the identified content item. The online system may apply the impression to update user profiles of the individuals, select broadcast content for presenting to the individuals, derive information describing attribution of the content items, or any combination thereof."

After wading through pages of practically undecipherable technical jargon, in basic terms, the technology functions as follows:

1.) a household has a broadcasting device through which content (i.e. advertising) is displayed

2.) each individual in a household has a smart device (i.e. smartphone, tablet or television) through which they access the internet

3.) the application detects whether the client device is in the vicinity of the broadcasting device and, if so, assumes that the individual is viewing content (i.e. advertising) on the broadcasting device

4.) when the broadcast signal is detected, the client device records the ambient audio of the content (i.e. advertising)

5.) the application extracts a hidden audio feature from the recorded ambient audio to generate identifying information about the content (think of it as an ambient audio fingerprint)

6.) the software then sends a data set including the ambient audio fingerprint, time information of the recorded ambient audio and an identifier of the individual  (i.e. you) to the online system

7.) the online system then determines whether the identified individual viewed the content item and then determines the impression associated with the content and the individual

8.) the online system may apply the impression to update the identified individual's user profile and may, in the future, select broadcast content for presenting content to the individual

Here is a diagram filed with the patent showing the basics of the system:

Here is one last quote from the filing (and remember to substitute "advertiser" or "advertising" for "content provider" or "content" respectively:

"Content providers have a vested interest in knowing who have listened and/or viewed their content and how many people have listened and/or viewed their content. Content providers have tried many ways to distribute their content to targeted audiences, e.g., TV broadcasting Over the Top (OTT) and Over the Air (OTA). Subsequent to presenting sponsored content provided by a content provider to users over the Internet and/or over TV broadcasting, the content providers want to tracks how often the users interact with the presented content and calculates statistics for the content. These statistics may be accrued over numerous content campaigns and serve to measure the effectiveness of each content item in the campaign. Based on these statistics, the content provider can edit content items that perform poorly or alternatively choose to show content items that have performed very effectively."

Let's close with this screen capture showing Facebook's third quarter 2018 results:

On a year-over-year basis, Facebook's advertising revenue (which made up 98.6 percent of its revenue) was up 33 percent to $13.539 billion for the quarter.  Advertising revenue in 2017 totalled $39.94 billion (98.3 percent of total revenue), up 49 percent on a year-over-year basis.

As you can see, without advertising, Facebook would be essentially a worthless entity.  Given this fact and Facebook's recent patent application, it's pretty easy to draw a straight line between Facebook's bottom line and a further and highly invasive incursion into our private lives.