Doing my Homework with AI (again)
Over the past year, AI tooling has advanced dramatically. The tool I am going to show you today is capable of producing answers to a wide range of abstract tasks across many different domains. I’m still going to use it to do my homework.
About a year ago, I posted an article where I used Github Copilot to see if a student could theoretically use it to complete homework assignments. Github Copilot is an AI-based autocomplete tool that can write code in many languages based on comments and context from surrounding code. Copilot is developed by GitHub and OpenAI and is based on the GPT-3 language model.
ChatGPT is a new GPT-3 based AI tool also created by OpenAI that is aimed at providing even more functionality for a wider range of tasks. Today, we’ll see how far AI tooling has come by tackling the same problem, LetterInventory, the first homework assignment in the University of Washington’s CSE 143 course. You can take a look at last years article on Copilot for some more context on the assignment and it’s goals.
ChatGPT provides a chat-like interface in which you “talk” with the AI. Each query you provide is met with a response from the AI in about 5-10 seconds. Here’s a quick example, this is a real excerpt from ChatGPT:
Me: Knock knock
ChatGPT: Who’s there?
Me: boo
ChatGPT: Boo who?
Me: Why are you crying?
ChatGPT: I am a machine learning model and do not have the ability to cry. I am programmed to assist with providing information and answering questions to the best of my knowledge. Is there something I can help you with?
Charming.
Warmup
Before we start our assignment, let’s try some simple code samples like we did with Copilot. Whereas Copilot worked as a VSCode extension, the conversational model of ChatGPT allows you to elaborate on previous queries and responses to refine the AI’s results. Quotes may be truncated for this article.
Me: Write a function in python to reverse a string
ChatGPT: Here is one possible implementation of a function that reverses a string in Python:
def reverse_string(string):
return string[::-1]
This function takes a string as input and returns a new string that is the reverse of the original string. It does this by using slicing to extract all the characters in the string in reverse order and then concatenating them together to form the new string. Here is an example of how you could use this function:
input_string = "Hello, world!"
reversed_string = reverse_string(input_string)
print(reversed_string) # Output: "!dlrow ,olleH"
I hope this helps! Let me know if you have any questions.
We can refine our results with further queries:
Me: add some comments and type hints to the function
ChatGPT:
def reverse_string(string: str) -> str:
"""
Reverses a given string.
Args:
string (str): The string to be reversed.
Returns:
str: The reversed string.
"""
# Use slicing to reverse the string and return the result
return string[::-1]
Lets change the requirements and see how it adapts:
Me: I change my mind, I want the function in Rust instead
ChatGPT:
fn reverse_string(string: &str) -> String {
// Create a mutable string to hold the reversed string
let mut reversed = String::new();
// Iterate over the characters in the string in reverse order
for character in string.chars().rev() {
// Push each character onto the mutable string
reversed.push(character);
}
// Return the reversed string
reversed
}
Doing my homework!
Last time, I wanted to see how little code I could get away with writing, instead relying on copilot and comments as much as possible. In the Copilot article, I wrote the class definition, method headers, and comments for each method. I also had to prod Copilot once or twice when it got stuck.
Today I’ll just be copy-pasting the entire spec at once and seeing what comes out:
Me: Help me complete this assignment in Java, respond with just a single code block, don’t include any additional explanations:
In this programming assignment you will practice using arrays and classes. You are to implement a class called LetterInventory that can be used to keep track of an inventory of letters of the alphabet. The constructor for the class takes a String and computes how many of each letter are in the String. This is the information the object keeps track of (how many a’s, how many b’s, etc). It ignores the case of the letters and ignores anything that is not an alphabetic character (e.g., it ignores punctuation characters, digits and anything else that is not a letter). Your class should have the following public methods. Method Description LetterInventory(String data) Constructs an inventory (a count) of the alphabetic letters in the given string, ignoring the case of letters and ignoring any non-alphabetic characters. int get(char letter) Returns a count of how many of this letter are in the inventory. Letter might be lowercase or uppercase (your method shouldn’t care). If a nonalphabetic character is passed, your method should throw an IllegalArgumentException. … (truncated) …
This is just ~4000 characters directly from the assignment PDF as well as some extra instructions about formatting.
And here is the result…
ChatGPT:
public class LetterInventory {
private int[] counts;
private int size;
private static final int NUM_LETTERS = 26;
public LetterInventory(String data) {
counts = new int[NUM_LETTERS];
size = 0;
data = data.toLowerCase();
for (int i = 0; i < data.length(); i++) {
char c = data.charAt(i);
if (c >= 'a' && c <= 'z') {
counts[c - 'a']++;
size++;
}
}
}
public int get(char letter) {
if (letter >= 'a' && letter <= 'z' || letter >= 'A' && letter <= 'Z') {
return counts[Character.toLowerCase(letter) - 'a'];
}
throw new IllegalArgumentException();
}
public void set(char letter, int value) {
if (letter >= 'a' && letter <= 'z' || letter >= 'A' && letter <= 'Z') {
size -= counts[Character.toLowerCase(letter) - 'a'];
counts[Character.toLowerCase(letter) - 'a'] = value;
size += value;
if (value < 0) {
throw new IllegalArgumentException();
}
} else {
throw new IllegalArgumentException();
}
}
public int size() {
return size;
}
public boolean isEmpty() {
return size == 0;
}
public String toString() {
StringBuilder sb = new StringBuilder();
sb.append('[');
for (int i = 0; i < NUM_LETTERS; i++) {
for (int j = 0; j < counts[i]; j++) {
sb.append((char)('a' + i));
}
}
sb.append(']');
return sb.toString();
}
public LetterInventory add(LetterInventory other) {
LetterInventory sum = new LetterInventory("");
for (int i = 0; i < NUM_LETTERS; i++) {
sum.counts[i] = this.counts[i] + other.counts[i];
}
sum.size = this.size + other.size;
return sum;
}
public LetterInventory subtract(LetterInventory other) {
LetterInventory diff = new LetterInventory("");
for (int i = 0; i < NUM_LETTERS; i++) {
diff.counts[i] = this.counts[i] - other.counts[i];
if (diff.counts[i] < 0) {
return null;
}
}
diff.size = this.size - other.size;
return diff;
}
}
This looks… good!
Let’s write a quick sanity check to make sure this compiles and some of the methods works:
public class Main {
public static void main(String[] args) {
LetterInventory inventory = new LetterInventory("ChatGPT wuz here! :)");
System.out.println(inventory.toString()); // [aceeghhprttuwz]
inventory.set('x', 5);
LetterInventory other = inventory.subtract(new LetterInventory("ee"));
System.out.println(other); // [acghhprttuwxxxxxz]
}
}
Let that sink in for a minute… I didn’t write a single line of code for this assignment, I just pasted the spec into a free online tool and compiled the result. Like we saw with our warmup example, I could always elaborate to get ChatGPT to add comments, write unit tests, and make other modifications to satisfy any other constraints of the assignment.
This is incredible to me. Especially considering it uses the same language model as Copilot, the advancement made in the past year just shows how quickly the field of AI is progressing. Just like our Copilot exercise, it raises even more pressing questions about academic integrity. This model is not limited to coding either, I’ve successfully gotten it to produce satisfactory answers to all sorts of “homework” questions like:
- Summarize “The Great Gatsby” in 300 words.
- Integrate this function, show your work:
ln(x - 1) / x^2. - Compare and contrast the concepts of direct democracy and representative democracy in terms of their structures, processes, and outcomes.
In fact, ChatGPT generated some of those questions as well, I’ll let you guess which.
Lets see what ChatGPT thinks about itself:
Me: What are some pros and cons of AI assistants, format the response as a table.
ChatGPT: Here are some pros and cons for AI-based assistants:
Pros Cons - Can perform tasks quickly and accurately - May not understand complex or abstract tasks - Can operate 24/7 without needing breaks or vacation time - May not be able to handle unexpected situations or problems outside of its programmed capabilities - Can provide personalized experiences by learning an individual’s preferences and habits - May not be able to replicate the intuition, creativity, or empathy of a human - Can improve over time as it processes more data and becomes more sophisticated - May be expensive to develop and maintain, depending on the complexity of the AI and the amount of data it needs to process - Can reduce the need for human labor, potentially leading to cost savings for businesses - May raise ethical concerns, such as the potential for AI to displace human workers or the fear that AI could become too advanced and potentially pose a threat to humanity
I guess we can even outsource ethics to AI now ;)
I’ll leave you with this Haiku, courtesy of ChatGPT, and an accompanying painting by OpenAI’s DALLE-E 2
The ocean’s depths blue
Waves crash against the shoreline
My heart is at peace.
