"OpenAI responded to the criticism by saying they'll allow researchers access to Codex. But the application process is opaque: researchers need to fill out a form, and the company decides who gets approved. It is not clear who counts as a researcher, how long they need to wait, or how many people will be approved. Most importantly, Codex is only available through the researcher program “for a limited period of time” (exactly how long is unknown)."
OP here. Many people are reacting to the title of the paper. A few thoughts:
* The paper is 35 pages long and it's hard to convey its message in any single title. We make clear in the text that our point is not that predictive optimization should never be used.
* We do want the _default_ to change from predictive optimization being seen as the obvious way to solve certain social problems to being against it until the developer can address certain objections. This is also made clear in the paper.
* The title is a nod to a famous book in this area called "Against prediction". Most people in our primary target audience are familiar with that book, so the title conveys a lot of information to those readers. That's one reason we picked it.
* Despite its flaws, when might we want to use predictive optimization? Section 4 gets into this in detail.
I learned from one of the comments on my original post that many scholars have been saying this for a while, and that there's in fact a book that makes the same point!
OP here. The full title of this article is "Students are acing their homework by turning in machine-generated essays. Good."
The last word was edited out by the mods, presumably under the belief that it's clickbait. Unfortunately, the headline now sounds like I'm complaining about this development, whereas my post is about how it will force much-needed improvements to education and free students from the drudgery of pointless essays that ask them to regurgitate content (as opposed to essays that teach writing skills or critical thinking, which remain valuable).
The problem is that the title was linkbaity, especially with that "Good" at the end. For a mod to edit that out is routine HN moderation ("Please use the original title, unless it is misleading or linkbait" - https://news.ycombinator.com/newsguidelines.html)
Perhaps a better fix is to use a representative sentence from the article body. I've done that now.
Those things are also clickbait and we also edit them out when we see them. Are you perhaps assuming that we see everything that gets posted here? That would be very mistaken.
>whereas my post is about how it will force much-needed improvements to education and free students from the drudgery of pointless essays that ask them to regurgitate content (as opposed to essays that teach writing skills or critical thinking, which remain valuable)
OK ... what's wrong with essays that demonstrate knowledge of a particular topic (as you call it "regurgitate content')? And why wouldn't those teach writing skills?
Yes, developing critical thinking skills is important, and sometimes you want to focus projects, assignment, homework towards that end. But don't discount the value of being able to synthesize and summarize existing knowledge in a particular area of knowledge. In fact, that's almost always a pre-requisite to making cogent arguments that exercise 'critical thinking' skills.
>In fact, it seems to be this kind of essay where language models are doing particularly well, with assignments such as “Write five good and bad things about biotech”. As an educator, I think this assignment is close to useless if the goal is to learn about biotech.
WHY?? Why is it 'close to useless' for a student to investigate current issues in biotech?
And by the way, with academics (and especially in public education), it is almost always the case that a student gets out of it what they put in. That is, if the student is aiming for the absolute minimum and takes every shortcut, neither AI, nor the assignment structure will a make a difference. Going back to this 'close to useless' question, a keen student can really sink their teeth into it and make this topic their own - because this question obviously is open-ended, and leaves room for the student to provide an independent and critical evaluation of the issues that concern the field .... OR ... they can spend 15 mins googling around or using AI, to throw a bunch of stuff together, call it a day, and go back to playing Call of Duty.
IMO I didn't get the sense that the title is complaining. I took the title as neutral statement and assumed the article would provide evidence for the claim and possibly commentary on it.
OP here. I totally agree that ideally authors should report most of this information in the paper itself. One advantage of a standalone document (we suggest putting it in an appendix) is that it's easy for reviewers to check that all of this information has been reported. Of course, authors could answer some of the questions by pointing to the sections of the paper in which they have been answered.
It's possible you may have misunderstood the title of the post. It isn't about the science of ML, or GPT-3, or brains. Rather, it's about using ML as a tool to do actual science, like medicine or political science or chemistry or whatnot. The first sentence of the post explains this.
Princeton University Center for Information Technology Policy | Princeton, NJ | Onsite | Full Time
Princeton CITP is a leading research center at the intersection of technology and public policy. We've conducted groundbreaking work on privacy, government surveillance, net neutrality, algorithmic fairness, dark patterns, and other high-profile topics. https://citp.princeton.edu/
We're hiring a data scientist who will collaborate with our world-class faculty, fellows, and students on interdisciplinary research projects and policy impact. If you live in New York City or New Jersey, are passionate about the societal impact of technology, and have an impressive resume in data science (broadly conceived), we want to hear from you.
That's fair. We don't claim that this is a new problem; we are merely adding evidence and our perspective to a known problem. We do link to others who have reported similar problems when trying to disclose vulnerabilities. The sentence saying we "discovered two wider issues" was worded poorly; in the paper [1] we used the word "encountered", and I've now edited the post to use the same wording. Thanks!
Just as important, the post is a PSA that there are 9 websites whose users remain vulnerable, and people with accounts on these sites should check their 2FA and password recovery settings. The websites are: Amazon, AOL, Finnair, Gaijin, Mailchimp, PayPal, Venmo, Wordpress.com, and Yahoo.
This is all just message board kibitzing! The blog post is good. I'm just conditioned by other message board threads on this problem. Thanks for writing it.
Princeton University Center for Information Technology Policy | Princeton, NJ | Onsite | Full Time
Princeton CITP is a leading research center at the intersection of technology and public policy. We've conducted groundbreaking work on privacy, government surveillance, net neutrality, algorithmic fairness, dark patterns, and other high-profile topics. https://citp.princeton.edu/
We're hiring a data scientist who will collaborate with our world-class faculty, fellows, and students on interdisciplinary research projects and policy impact. If you live in New York City or New Jersey, are passionate about the societal impact of technology, and have an impressive resume in data science (broadly conceived), we want to hear from you.
Author here. I appreciate your criticism. What I had in mind was more along the lines of Google's claims around diabetic retinopathy. I received feedback very similar to yours, i.e. that those claims are based on an extremely narrow problem formulation: https://twitter.com/MaxALittle/status/1196957870853627904
I will correct this in future versions of the talk and paper.
Then I shall write to you directly. I don’t know how you can make the claim that automated essay grading is anything but a shockingly mendacious academic abuse of student’s time and brainpower. To me, this seems far worse than job applicant filtering, firstly because hiring is fundamentally predictive, and secondly because many jobs have a component of legitimately rigid qualifications. An essay is a tool to affect the thoughts of a human. It is not predictive of some hidden factor; it stands alone. It must be original to have value; a learned pattern of ideas is the anti-pattern for novelty. If the grading of an essay can be, in any way, assisted by an algorithm, it is probably not worth human effort to produce. If you personally use essay grading software, or know of anybody at Princeton that does, you have an absolute obligation to disclose this to all of your students and prospective applicants. They are paying for humans to help them become better humans.
Thanks for the .pdf and the research in general, great stuff!
One thing I'd love is a look at 'noise' in these systems, specifically injecting noise into them. Addons like Noiszy [0] and trackmenot [1] claim to help, but I'd imagine that doing so with your GPS location is a bit tougher. I'd love to know more on such tactics, as it seems that opt-ing out of tracking isn't super feasible anymore (despite the effectiveness of the tracking).
"OpenAI responded to the criticism by saying they'll allow researchers access to Codex. But the application process is opaque: researchers need to fill out a form, and the company decides who gets approved. It is not clear who counts as a researcher, how long they need to wait, or how many people will be approved. Most importantly, Codex is only available through the researcher program “for a limited period of time” (exactly how long is unknown)."