Katharina Brunner

Machine Learning

Mit Daten arbeiten und ihnen gegenüber skeptisch sein, ist kein Widerspruch – im Gegenteil

by Kommentar verfassen

I’m a data scientist who is skeptical about data schreibt Andrea Jones-Rooy bei Quartz. Da gibts viel zu zitieren:

Whether it’s curing cancer, solving workplace inequality, or winning elections, data is now perceived as being the Rosetta stone for cracking the code of pretty much all of human existence.

But in the frenzy, we’ve conflated data with truth. And this has dangerous implications for our ability to understand, explain, and improve the things we care about.

(…)

“What does the data say?”

Data doesn’t say anything. Humans say things. They say what they notice or look for in data—data that only exists in the first place because humans chose to collect it, and they collected it using human-made tools.

Data can’t say anything about an issue any more than a hammer can build a house or almond meal can make a macaron. Data is a necessary ingredient in discovery, but you need a human to select it, shape it, and then turn it into an insight.

(…)

Data is an imperfect approximation of some aspect of the world at a certain time and place.

(…)
Companies—and my students—are so obsessed with being on the cutting edge of methodologies that they’re skipping the deeper question: Why are we measuring this in this way in the first place? Is there another way we could more thoroughly understand people? And, given the data we have, how can we adjust our filters to reduce some of this bias?

(…)

This doesn’t mean throw out data. It means that when we include evidence in our analysis, we should think about the biases that have affected their reliability. We should not just ask “what does it say?” but ask, “who collected it, how did they do it, and how did those decisions affect the results?”

We need to question data rather than assuming that just because we’ve assigned a number to something that it’s suddenly the cold, hard Truth. When you encounter a study or dataset, I urge you to ask: What might be missing from this picture? What’s another way to consider what happened? And what does this particular measure rule in, rule out, or incentivize?

Links concerning the Internet

by Kommentar verfassen

  • Building a data culture: „self-service learning program to facilitate fun, creative introductions for the non-technical folks in your organization“
  • verwandt dazu: Data Playbook: „The Data Playbook (Beta) builds on social learning and modularized activities already developed to promote data literacy via workshops“
  • A manifesto for the Internet Age:

  • Immer wieder das gleiche Problem: Wer selbstlernende Algorithmen braucht, muss vorher Daten in guter Qualität haben. Bei der Gesichtserkennung heißt das: Viele, viele, viele Fotos mit Angaben zu Hautfarbe, Alter, Geschlecht und einer Menge anderer Eigenschaften. Und woher bekommen Firmen und Forscherinnen diese Bilder? Zum Beispiel durch Scrapen oder Bulk-Dateien der ehemaligen Foto-Plattform Flickr. Das hat zwei Probleme: Erstens ein Einbruch in die Privatssphäre. Zweitens können die Bilder dafür verwendet werden, Überwachungssoftware zu trainieren. Mehr: Facial recognition’s ‚dirty little secret‘: Millions of online photos scraped without consent

Die Zukunft suchen und nicht finden

by Kommentar verfassen

Vorhersagen auf Basis von Berechnungen sind ein Spiel mit der Zeit.

Die Polizei ist auf einer Art Zeitreise. Sie will Verdächtige schon möglichst früh ausmachen – und das nicht nur im Flugverkehr. „Vor die Lage kommen“, nannte das der ehemalige BKA-Chef Jörg Ziercke.

Morgen ein Mörder

Und außerdem: Das Problem mit den Falsch-Positiven (SZ). Bei Prognose-Algorithmen kommt es nicht nur darauf an, welche Personen richtig erkannt werden, sondern auch darauf, welche Personen zu unrecht verdächtigt werden. Mathematisch nennt man diesen Fehler bei binären Entscheidungen (ja – nein, verdächtig, unverdächtig) die false-positve rate.

Bei der Gesichtserkennung am Berliner Bahnhof Südkreuz gibt die Polizei eine Falschtrefferrate von 0,1 Prozent an. Klingt wenig, die Zahl kleiner 1 führt aber in die Irre. In Wahrheit ist das ein unglaublich hoher, ein zu hoher Wert. Vanessa Wormer und Christian Endt rechnen das in der SZ vor:

  • Etwa 12 Millionen Bahnfahrer pro Tag
  • eine Falschpositivrate von 0,1 Prozent ergibt 12 000 unschuldig Verdächtige

Das ist das Problem an derartigen Systemen, die auf anlassloser Massenüberwachung fußen: Selbst bei sehr geringen Fehlerraten geraten ungleich viele Personen fälschlicherweise ins Visier der Fahnder.

Viele Daten, nix dahinter

by Kommentar verfassen

Also, dieses Maschinelle Lernen ist ja überall. Aber ist es auch überall notwendig? Oder sind die Ergebnisse oftmals, nun ja, erwartbar. Und mit ein bisschen nachdenken und präzisen Algorithmen nicht mindestens genauso zu erreichen?

This is, by the way, the dirty secret of the machine learning movement: almost everything produced by ML could have been produced, more cheaply, using a very dumb heuristic you coded up by hand, because mostly the ML is trained by feeding it examples of what humans did while following a very dumb heuristic. There’s no magic here. If you use ML to teach a computer how to sort through resumes, it will recommend you interview people with male, white-sounding names, because it turns out that’s what your HR department already does. If you ask it what video a person like you wants to see next, it will recommend some political propaganda crap, because 50% of the time 90% of the people do watch that next, because they can’t help themselves, and that’s a pretty good success rate.

Das Zitat stammt aus einem Blogpost der Gattung „schöne Mischung aus Rant und Analyse“ und hat die These: Für Empfehlungsalgorithmen braucht’s jetzt dieses viele Daten sammeln eigentlich wirklich nicht.

Mehr aus Forget privacy: you’re terrible at targeting anyway:

Probably what it does is infer my gender, age, income level, and marital status. After that, it sells me cars and gadgets if I’m a guy, and fashion if I’m a woman. Not because all guys like cars and gadgets, but because some very uncreative human got into the loop and said „please sell my car mostly to men“ and „please sell my fashion items mostly to women.“ Maybe the AI infers the wrong demographic information (I know Google has mine wrong) but it doesn’t really matter, because it’s usually mostly right, which is better than 0% right, and advertisers get some mostly demographically targeted ads, which is better than 0% targeted ads.


You know this is how it works, right? It has to be. You can infer it from how bad the ads are. Anyone can, in a few seconds, think of some stuff they really want to buy which The Algorithm has failed to offer them, all while Outbrain makes zillions of dollars sending links about car insurance to non-car-owning Manhattanites. It might as well be a 1990s late-night TV infomercial, where all they knew for sure about my demographic profile is that I was still awake.

KI, ML, etc.: Es dauert noch und das ist ganz normal so

by Kommentar verfassen

AI hier, AI da. Aber wer setzt es wirklich ein? MIT Technology Review macht den Realitätscheck und bestätigt vieles derjenigen, die sich praktisch damit befassen.

It’s one thing to see breakthroughs in artificial intelligence that can outplay grandmasters of Go, or even to have devices that turn on music at your command. It’s another thing to use AI to make more than incremental changes in businesses that aren’t inherently digital.

Denn Google, Amazon, Facebook, Netflix und die anderen großen Firmen haben extrem viele Mitarbeiter, die sich nur damit befassen und das Geschäftsmodell ist inheränt auf Daten ausgelegt. In anderen Branchen ist das nicht so.

Data scientists at IBM and Fluor didn’t need long to mock up algorithms the system would use, says Leslie Lindgren, Fluor’s vice president of information management. What took much more time was refining the technology with the close participation of Fluor employees who would use the system. In order for them to trust its judgments, they needed to have input into how it would work, and they had to carefully validate its results, Lindgren says.


To develop a system like this, “you have to bring your domain experts from the business—I mean your best people,” she says. “That means you have to pull them off other things.” Using top people was essential, she adds, because building the AI engine was “too important, too long, and too expensive” for them to do otherwise.

Es wird also, so das Fazit, noch etwas dauern, bis Künstliche Intelliganz und Maschinelles Lernen auch in nicht-Tech-Branchen in der Breite ankommt. Ungewönlich ist das nicht:

What (…) economists confirmed, is that the spread of technologies is shaped less by the intrinsic qualities of the innovations than by the economic situations of the users. The users’ key question is not, as it is for technologists, “What can the technology do?” but “How much will we benefit from investing in it?”

Links:

An economist’s view on AI

by 1 Kommentar

Michaela Schmöller, writes in „Secular stagnation: A false alarm in the euro area?“ for the Bank of Finland

It is important to note that productivity growth evolves in a two-stage process: the initial invention of new technologies through research and development, subsequently followed by technological diffusion, i.e. the incorporation of these new technologies in the production processes of firms. As a result, even though many important technology advances may have been invented in recent times, they will only exert an effect on output and productivity once firms utilise these technologies in production. Potential productivity gains from technologies that have yet to be widely adopted may be sizable. A central example is the field of artificial intelligence in which future productivity gains may be considerable once AI-related technologies diffuse to the wider economy. (…)

AI may represent — as did the steam engine, the internal combustion engine and personal computers — a general purpose technology, meaning that it is far-reaching, holds the potential for further future improvements and has the capability of spurring other major, complementary innovations over time with the power of drastically boosting productivity. Incorporating AI in production requires substantial changes on the firm-level, including capital stock adjustments, the revision of internal processes and infrastructures, as well as adapting supply and value chains to enable the absorption of these new technologies. Consequently, this initial adjustment related to the incorporation of general purpose technologies in firms‘ production may take time and may initially even be accompanied by a drop in labour productivity before delivering positive productivity gains.