My Work

<aside> 💡 Explore ImageNet. ImageNet sample images, Kaggle ImageNet Mini 1000, What surprises you about this data set? What questions do you have? Thinking back to last week’s assignment, can you think of any ethical considerations around how this data was collected Are there privacy considerations with the data?

</aside>

Analyzing the ImageNet dataset closely reminds me of the saying “Innovation before Regulation”

1. Surprises: Before studying the dataset closely, I thought it would contain ‘better quality’ images. Or that there would be some guideline in place to regulate which types of images to include. Here are some observations that made me question my theory:

   1. I noticed that sometimes the images were in black and white or parts of the ‘object’ were cut off.
   2. It was also sometimes hard to identify which ‘item’ in the image was the target item (for example: the image for ‘plate’ is covered in a bunch of food’). This was especially difficult when items were taken apart into separate components (ex. power drill)
   3. Some images were blurry, grainy or otherwise low quality

2. Privacy and Ethical Concerns:

   1. It is clear that the dataset consists of images from a variety of sources.
      1. Some seemed more ‘professional’ meaning they might have been copyrighted or used for profit elsewhere. (‘canoe’ seems to be taken from a company’s brochure with Chinese words on it or plate with DCH watermark)
      2. Others seemed like pictures people post on their personal social media.
      3. Either way, all these images were presumably made public onto the internet so legally it may be okay to use them. However, whether or not it is ethical is story.
   2. Many images featured humans, even if the target object is not the human (eg. bow_tie). It is unclear whether these people gave consent to have their images be made public or used to train ml models.
      1. Some people in the images even looked confused, as if they didn’t know they were getting their pictures taken? (eg. grocery store)
   3. Culturally specific objects are featured: Who got to decide which pictures could accurately identify cultural items? (eg. Yurt).
      1. Since these images are gathered from the internet, they could be inaccurate or made by people purposefully being offensive. This could cause the resulting model to perpetuate harmful stereotypes.
   4. Dataset objects are biased towards some and does not accurately reflect others
      1. Example: A lot of American animals. Perhaps these are very common animals, but what about equally common animals in other parts of the world (eg. American Staffordshire Terrier)
      2. The dataset also consists only images posted by people who have access to the internet and modern technologies. This means the opinions of those who do not have these privileges are not accounted for. This is similar to the policing scenario from first weeks reading.

<aside> 💡 Using the ml5.js examples above, try running image classification on a variety of images. Pick at least 10 objects in your room. How many of these does it recognize? What other aspects of the image affect the classification, including but not limited to position, scale, lighting, etc.

</aside>

Results Analysis:

Out of all the individual items I tested under good lighting and with a white background (not including multiple object experiments)… only about 3/10 (Granny Smith, Mug, Colander) were consistently identified with 1 being occasionally identified (Ball-point Pen).

Factors that impact classification:

1. Amount of screen the target item takes up (scale): Sometimes imageClassifier can’t figure out witch item on the screen to ‘identify’. I found that close up shots or increasing the portion of screen the object takes up yields better results.
2. Lighting: The model worked better in well lit environments. I tried detecting the same object in 2 different lights and observed that the model was more consistent when used with brighter lighting.
3. Background: When the background has a messy patterns or have a lot of different objects and colors, imageClassifier will flicker between different classifications, and when it is correct, the probability is low. Holding the object against a solid color background worked best.
4. Orientation: As with my test of the mug, It seems that uncommon orientation can cause lower detection accuracy. Perhaps it’s because the dataset does not have enough samples from different orientations.