There is so much wrong in this article, The Real Bias Built In at Facebook – The New York Times, that I decided to take it apart in this blog post. (I’ve read so many bad IT stories in the Times that I stopped critiquing them after a while, but this one in particular bugged me enough to write something).
To illustrate what I mean by what is wrong with this piece, here’s some excerpts in italics followed by my thoughts in non-italics.
- First off, there is the use of the word “algorithm” everywhere. That alone is a problem. For an example of why that is bad, see section 2.4 of Paul Ford’s great piece on software,What is Code? As Ford explains: ““Algorithm” is a word writers invoke to sound smart about technology. Journalists tend to talk about “Facebook’s algorithm” or a “Google algorithm,” which is usually inaccurate. They mean “software.” Now part of the problem is that Google and Facebook talk about their algorithms, but really they are talking about their software, which will incorporate many algorithms. For example, Google does it here: https://webmasters.googleblog.com/2011/05/more-guidance-on-building-high-quality.html At least Google talks about algorithms, not algorithm. Either way, talking about algorithms is bad. It’s software, not algorithms, and if you can’t see the difference, that is a good indication you should not be writing think pieces about I.T.
- Then there is this quote: “Algorithms in human affairs are generally complex computer programs that crunch data and perform computations to optimize outcomes chosen by programmers. Such an algorithm isn’t some pure sifting mechanism, spitting out objective answers in response to scientific calculations. Nor is it a mere reflection of the desires of the programmers. We use these algorithms to explore questions that have no right answer to begin with, so we don’t even have a straightforward way to calibrate or correct them.” What does that even mean? To me, I think it implies any software that is socially oriented (as opposed to say banking software or airline travel software) is imprecise or unpredictable. But at best, that is only slightly true and mainly false. Facebook and Google both want to give you relevant answers. If you start typing in “restaurants” or some other facilities in Google search box, Google will start suggesting answers to you. These answers will very likely to be relevant to you. It is important for Google that this happens, because this is how they make money from advertisers. They have a way of calibrating and correcting this. In fact I am certain they spend a lot of resources making sure you have the correct answer or close to the correct answer. Facebook is the same way. The results you get back are not random. They are designed, built and tested to be relevant to you. The more relevant they are, the more successful these companies are. The responses are generally right ones.
- “ If Google shows you these 11 results instead of those 11, or if a hiring algorithm puts this person’s résumé at the top of a file and not that one, who is to definitively say what is correct, and what is wrong?” Actually, Google can say, they just don’t. It’s not in their business interest to explain in detail how their software works. They do explain generally, in order to help people insure their sites stay relevant. (See the link I provided above). But if they provide too much detail, bad sites game their sites and make Google search results worse for everyone. As well, if they provide too much detail, they can make it easier for other search engine sites – yes, they still exist – to compete with them.
- “Without laws of nature to anchor them, algorithms used in such subjective decision making can never be truly neutral, objective or scientific.” This is simply nonsense.
- “Programmers do not, and often cannot, predict what their complex programs will do. “ Also untrue. If this was true, then IBM could not improve Watson to be more accurate. Google could not have their sales reps convince ad buyers that it is worth their money to pay Google to show their ads. Same for Facebook, Twitter, and any web site that is dependent on advertising as a revenue stream.
- “Google’s Internet services are billions of lines of code.” So what? And how is this a measure of complexity? I’ve seen small amounts of code that was poorly maintained be very hard to understand, and large amounts of code that was well maintained be very simple to understand.
- “Once these algorithms with an enormous number of moving parts are set loose, they then interact with the world, and learn and react. The consequences aren’t easily predictable. Our computational methods are also getting more enigmatic. Machine learning is a rapidly spreading technique that allows computers to independently learn to learn — almost as we do as humans — by churning through the copious disorganized data, including data we generate in digital environments. However, while we now know how to make machines learn, we don’t really know what exact knowledge they have gained. If we did, we wouldn’t need them to learn things themselves: We’d just program the method directly.” This is just a cluster of ideas slammed together, a word sandwich with layers of phrases without saying anything. It makes it sound like AI has been unleashed upon the world and we are helpless to do anything about it. That’s ridiculous. As well, it’s vague enough that it is hard to dispute without talking in detail about how A.I. and machine learning works, but it seems knowledgeable enough that many people think it has greater meaning.
- “With algorithms, we don’t have an engineering breakthrough that’s making life more precise, but billions of semi-savant mini-Frankensteins, often with narrow but deep expertise that we no longer understand, spitting out answers here and there to questions we can’t judge just by numbers, all under the cloak of objectivity and science.” This is just scaremongering.
- “If these algorithms are not scientifically computing answers to questions with objective right answers, what are they doing? Mostly, they “optimize” output to parameters the company chooses, crucially, under conditions also shaped by the company. On Facebook the goal is to maximize the amount of engagement you have with the site and keep the site ad-friendly.You can easily click on “like,” for example, but there is not yet a “this was a challenging but important story” button. This setup, rather than the hidden personal beliefs of programmers, is where the thorny biases creep into algorithms, and that’s why it’s perfectly plausible for Facebook’s work force to be liberal, and yet for the site to be a powerful conduit for conservative ideas as well as conspiracy theories and hoaxes — along with upbeat stories and weighty debates. Indeed, on Facebook, Donald J. Trump fares better than any other candidate, and anti-vaccination theories like those peddled by Mr. Beck easily go viral. The newsfeed algorithm also values comments and sharing. All this suits content designed to generate either a sense of oversize delight or righteous outrage and go viral, hoaxes and conspiracies as well as baby pictures, happy announcements (that can be liked) and important news and discussions.” This is the one thing in the piece that I agreed with, and it points to the real challenge with Facebook’s software. I think the software IS neutral, in that it is not interested in the content per se as it is how the user is responding or not responding to it. What is NOT neutral is the data it is working off of. Facebook’s software is as susceptible to GIGO (garbage in, garbage out) as any other software. So if you have a lot of people on Facebook sending around cat pictures and stupid things some politicians are saying, people are going to respond to it and Facebook’s software is going to respond to that response.
- “Facebook’s own research shows that the choices its algorithm makes can influence people’s mood and even affect elections by shaping turnout. For example, in August 2014, my analysis found that Facebook’s newsfeed algorithm largely buried news of protests over the killing of Michael Brown by a police officer in Ferguson, Mo., probably because the story was certainly not “like”-able and even hard to comment on. Without likes or comments, the algorithm showed Ferguson posts to fewer people, generating even fewer likes in a spiral of algorithmic silence. The story seemed to break through only after many people expressed outrage on the algorithmically unfiltered Twitter platform, finally forcing the news to national prominence.” Also true. Additionally, Facebook got into trouble for the research they did showing their software can manipulate people by….manipulating people in experiments on them! It was dumb, unethical, and possibly illegal.
- “Software giants would like us to believe their algorithms are objective and neutral, so they can avoid responsibility for their enormous power as gatekeepers while maintaining as large an audience as possible.” Well, not exactly. It’s true that Facebook and Twitter are flirting with the notion of becoming more news organizations, but I don’t think they have decided whether or not they should make the leap or not. Mostly what they are focused on are channels that allow them to gain greater audiences for their ads with few if any restrictions.
In short, like many of the IT think pieces I have seen the Times, it is filled with wrong headed generalities and overstatements, in addition to some concrete examples buried somewhere in the piece that likely was thing that generated the idea to write the piece in the first place. Terrible.
Things I am interested in or working on these days: AI, WebSphere setup, Python, Twitter programming, development in general, configuring Netscalers, cool things IBM is doing, automation, among other things.
- If you have the AI bug and think you want to do some Prolog programming, you need this: What Prolog implementation to choose? What’s fastest? Compatibility?
- Deep Learning is hot in AI. If you want more info, this is good: Deep Learning Tutorials — DeepLearning 0.1 documentation
- Sigh. This debate never goes away in AI: Why AlphaGo Is Not AI – IEEE Spectrum
- More on the hysteria that AI brings: The founder of Evernote made a great point about why AI (probably) won’t kill us all – Vox
- Ignore most AI hysteria, but do read this: What does it mean for an algorithm to be fair? | Math ∩ Programming
- Want to whip up a quick mobile app? Consider: Mobile App Builder – new service now available – Bluemix Blog
- For power users, there’s: How to create an insane multiple monitor setup with three, four, or more displays | PCWorld
- Need virtual images? Take a look at this: Images | VirtualBoxes – Free VirtualBox® Images
- For hardcore WAS users, this is helpful: Installing optional Java 7.x on WebSphere Application Server 8.5 (Application Integration Middleware Support Blog)
- A classic. Anyone tuning WAS needs this: Case study: Tuning WebSphere Application Server V7 and V8 for performance
- Want to learn Python? Write your own Twitter client? Or do both? Then there’s this: How To Build a Twitter “Hello World” Web App in Python | ProgrammableWeb
- More on programming Twitter: How To Use The Twitter API To Find Events | ProgrammableWeb
- Nice little project to try, here: Create a mobile-friendly to-do list app with PHP, jQuery Mobile, and Google Tasks
- Creating Simple Responsive HTML5 and PHP Contact Form | Future Tutorials
- Setting up a Linux system? Then you want to read this: Most secure way to partition linux? – Information Security Stack Exchange
- Want to learn Linux? This is essential! IBM developerWorks : Technical library concerning Learning Linux
- If you are doing performance work on Unix, you will likely use vmstat. Even if you know vmstat, this is good to review: What to look for in vmstat – UNIX vmstat command
- Wow! OS/2 is still alive! OS/2: Blue Lion to be the next distro of the 28-year-old – Yahoo Finance
- Talk about old tech! This makes OS/2 seem fresh! It’s Insane that New York’s Subway Still Runs on This 80-Year-Old Switchboard | Motherboard
- I was doing some work on Netscaler and found this useful in comparing the set up of one Netscaler config with another: Export Netscaler Config – NetScaler Application Delivery – Discussions. This is also useful: Netscaler 9 Cheat Sheet.doc – netscaler9cheatsheet.pdf
- I thought this was a good development for everyone interested in Node: IBM Buys StrongLoop To Add Node.js API Development To Its Cloud Platform | TechCrunch
- Alot has changed with IBM’s OpenPOWER. Forbes gets you up to date, here: IBM’s OpenPOWER: A Lot Has Changed In Two Years – Forbes
- Cool stuff here: Access your Docker-based Raspberry Pi at home from the internet · Docker Pirates ARMed with explosive stuff
- I was using Perl scripts on Linux to send me messages to my mobile device via Pushover. This was good for that: pushover Archives – Perl Hacks
- I was also using WinSCP for that and this helped: Scripting and Task Automation :: WinSCP
- For all those trying to succeed in IT but feeling you are running into ceiling, you should read this: Tech’s Enduring Great-Man Myth or this When It Comes to Age Bias, Tech Companies Don’t Even Bother to Lie | Dan Lyons | LinkedIn
- Linus Torvalds is always interesting, and this is especially good: Linux at 25: Q&A With Linus Torvalds – IEEE Spectrum
- Very cool! Particle | Build your Internet of Things
- And finally some links to good stuff on UML online: Multi-layered web architecture UML package diagram example, web layer depends on business layer, which depends on data access layer and data transfer objects.
Posted in cool, IT
Tagged AI, IT, Linux, netscaler, OS2, performance, PHP, Prolog, Python, SoftLayer, twitter, WebSphere
A nice use of computing to refute an old conjecture from no less than Euler.
Found thanks to a tweet from
Posted in new!
Tagged Euler, math, twitter