ImageNet，Image Classifier

When I saw the image dataset, I found that there are many types of datasets, but some categories have very little data, and for image recognition, many images have similar content. This creates difficulties in distinguishing them. As stated in Excavating AI, many training sets are not precise and may contain biased, stereotypical, and discriminatory images. There are more white datasets than non white datasets. Additionally, I am also surprised why image recognition is not divided into several major categories. It's like not only identifying dog breeds, but also identifying them as dogs. Because in many cases, it is only necessary to know the major categories and not the detailed content.

Image Classification

In this project, I originally wanted to use a camera to allow users to find inspiration by holding objects around them. However, because the image classifier is not very accurate, I opted to use images instead. As a user-oriented tool, I wanted users to be able to upload photos via a button, but when I changed the photo upload method to be outside of p5, the image classifier errored, preventing me from slicing the first result in the object's label. Therefore, I had to continue completing the project by uploading within the p5 system. In this project, I utilized the ChatGPT API, extracting the most relevant first result from the labels to provide designers with some inspiration and suggestions for their designs.

let classifier;
let img;
let chatResponse = ""; 

const API_KEY = "YOUR API KEY";
const url = "<https://api.openai.com/v1/chat/completions>";
let options = {
    method: "POST",
    headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${API_KEY}`
    },
}

function preload() {
    img = loadImage("butterfly.png", () => {
        console.log('Image loaded');
    });
    classifier = ml5.imageClassifier("MobileNet", () => {
        console.log('Model loaded');
        classifier.classify(img, gotResults);
    });
}

function gotResults(results) {
    console.log(results);
    const fullLabel = results[0].label;
    const firstLabel = fullLabel.split(',')[0];
    console.log(firstLabel);
    fetchChatGPTSuggestions(firstLabel); 
}

function setup() {
    createCanvas(600, 600);
    textAlign(CENTER, CENTER);
    textSize(16);
    textStyle(BOLD);
    textFont('Helvetica');
}

function draw() {
    background(220);
    fill(0);
    textSize(20);
    text("Please upload one picture that you want to get idea for.", 50, 40);

    imageMode(CENTER);
    if (img) {
        image(img, width / 2, 230, 280, 280); 
    }
  
    textSize(16);
    fill(0);
    textAlign(LEFT, TOP);
    text("Designer Response: " + chatResponse, 10, 400, width-20); 
}

function fetchChatGPTSuggestions(firstLabel) {
    options.body = JSON.stringify({
        model: "gpt-3.5-turbo-0125",
        messages: [
            {
                role: "system",
                content: `Give ideas for what "${firstLabel}" can create.`
            },
            {
                role: "user",
                content: "Could you give me some advice?"
            },
        ],
        temperature: 0.8,
        max_tokens: 50,
        top_p: 1,
        frequency_penalty: 0,
        presence_penalty: 0.8,
    });
    fetch(url, options)
        .then((response) => {
            if (!response.ok) {
                throw new Error('Network response was not ok');
            }
            return response.json();
        })
        .then((response) => {
            if (response.choices && response.choices.length > 0) {
                chatResponse = response.choices[0].message.content;
            } 
        })
}

In this project, I saw the limitations of both the image classifier and AI, as neither are very precise. For instance, when I wanted the photo to recognize a white pleated skirt, it identified it as a lantern shape. The result is technically correct, but we would not normally perceive it as a lantern shape.