Tech Talk: Automated Content Analysis — Talking Tech W/ Jasmine McNealy & Dhanaraj Thakur

Hi, I'm Riddhi Shetty. I work on the Privacy and Data Project here at CDT. Recently, we've been advocating for stronger federal and state guidance and regulations against consumer data harms that limit economic opportunity.

You can support this and all we do here at CDT by going to CDT.org slash tech talk and donating. Every donation matters. Thank you for enhancing civil rights and civil liberties in the digital age.

Welcome to Tech Talk. Bye. C-D-T. Welcome to C-D-T's Tech Talk, where we dish on tech and internet policy while also explaining

what these policies mean to our daily lives. I'm Jamal Maghvi and it's time to talk tech. Here to help us understand more about the capabilities and limitations of automated content analysis, our Jasmine McNeely, CDT non-residential fellow and associate professor in the Department of

Telecommunication, College of Journalism and Communications at the University of Florida, and Donorosh Stacker, Research Director for CDT. Jasmine and Donorosh, thank you so much for joining us today. All right, so to kick us off, can you both explain what automated content analysis

is? Maybe I could start if that's okay. And thank you, Jamal, for hosting this talk. I think the topic is very relevant, right?

Automated content analysis has become even particularly relevant in recent times. I mean, there are statistics that suggest that, there are like three billion images

that are uploaded every day on YouTube, maybe even as much as 500 hours of video, a minute are created, right? So basically because of the shared scale of content

has been uploaded in addition to like increased calls among governments and policymakers to restrict particular kinds of content, there's been an increased use of these kinds of tools

to automatically detect content of particular nature to inform moderation decisions on social network and on other services. So these kinds of tools and understanding what these tools are

and your capabilities and limitation is something that we are particularly concerned with. And we argue that it's really important that other stakeholders like policymakers

and companies themselves understand what's happening here as well. Right, so I agree with Donaraj. I think it's a collection of techniques

that allow for the analysis of text of images, of video, so all different kinds of media that can be found in a digital, so to speak format or digitized format that can then be used

for things like content moderation, but also things that marketers want to know about like sentiment analysis and other kinds of ways of predicting or attaching meaning to the expression.

the expression, so the visuals, the text, the video, all kinds of things. So being able to label it or attach meaning to it automatically or using a set of tools or techniques

and then being able to make that data available for whatever purpose that an organization or a government or a group wants to use it for. Now, from what I understand,

there are both matching and predictive models. Can you explain what these are and how they're used? Yeah, sure. I think, just following what Jasmine just said

in terms of these different kinds of, the range of techniques, matching and predictive models are a way of this generally grouping these kinds of different techniques.

Most of these techniques rely on some form of machine learning and that's essentially a means to like parse through and analyze large amounts of data to identify characteristics or relationships or correlations

within that data that really are relevant to the objective of the model, right? So that could mean like identifying images or particular sounds or video

that is that developers are interested in. And practically speaking, machines can make these kinds of identifications or labeling in two broad ways.

One is matching. So essentially recognizing something that's identical or similar to something it has seen before. And prediction, which is recognizing the characteristics

of or features of a piece of content that's based on the machine's prior learning, right? And learning from a large amount of data. So matching is basically,

have I seen this image or audio a bit before? And prediction is really, to what extent does it fit the characteristics of an image or audio that I want to identify.

Okay, and I would add for prediction, the thing about prediction is that the machine can infer values to form missing values. So if there's enough of one kind of characteristic

or a set of characteristics that reminds it of something it's seen before, then the predictive models that can say, this will probably end up more like this past thing,

even if I don't have the value for these certain characteristics. The other characteristics I do have the value for, tell me it's leaning more towards this label or this category or this kind of inference

about whatever it is it's making an inference about. Yeah. You know what Jasmine is saying makes me think of, if we're thinking of like a practical example, right?

Like say, you want a model to tell you whether an image contains a cat or not. And so you could have a classifier that's developed that way. Essentially, it's making a prediction.

Based on previous data it has learned from, does the image contain its cat? And it often presents that analysis in the form of a prediction.

And that's how often, like when it comes to these predictive techniques, that's how it's often, in a very simple way, that's how it often works. So it's looking at different kinds of shapes and textures

and colors that are relevant. But as Jasmine is saying, sometimes a model might not have all information, but for a particular variable, like for what's the color that it's looking for,

or what the shape or so on. And so it kind of will try and fill the gap. But ultimately, it's making a prediction. And that's what we have to be clear about, right?

That it's making an effect on educated guests as to whether this is what we are looking for. Jasmine, let me go back to you and I ask, what are some of the limitations

to automated content analysis?