Thanks to Sao Ohtake for handmodelling

Continuing from few weeks ago...

a recap

Inspired by the contemporary audience behaviour in art gallery and exhibition, where audience are carrying another set pair of eyes -- their mobile device camera -- to take picture of the work. Some people may not even look at the work for as long as their camera takes the picture.

By noticing the property of the aliasing effect that happens on camera due to its specific range of shutter speed and/or fps, we can take advantage of this to insert our hidden message in the artwork.

The project could be either showing 2 different versions of the artwork (e.g, the artwork shows A when seen with eye, and it shows pattern B when seen through camera, but both carry different but meaningful information such as text or different shape), or, it could really be substracting the experience (e.g, the face got sensored when it is seen through a camera, or it reveals a watermarked version of the work so that people will by default credit me if they circulate this image of the artwork).

Technology Used

- p5js: ml5's FaceMesh, Serial Communication

- Arduino: flicker rate, parsing pixels from p5js

Materials

- 3 piece of 8x24 neopixel strips, not daisy-chained so i can return it, connected to 3 different digital pins

- 1000 mf capacitor

- 3 piece of 470 Ohm resistors, connected to each neopixel strips

- 5V power source

- Matte acrylic & frosted acrylic, cardboard

Parsing Frame Logic

Technology Exploration

I hit a plateau as my Arduino Nano 303 IoT couldn't get into the realm of micros to update the flicker of the LEDs (from HIGH to LOW). I consulted ChatGPT first and it suggested ESP32, however, when I tried to reconfigure the whole circuit with a Nano + ESP32, it couldn't even get the Neopixels to light up. Some considerable time is spent on this, and I came to a point where it is not worth it to pursue with this microcontroller. Just as I was thinking to downgrade the project to use Nano 303 and just use 1 strip of NeoPixel with text on it (my initial idea was to put an image, and even more ambitious, a camera stream).

Coming into rescue, Fabri suggested me to try Teensy 4.0 as apparently it is the fastest microcontroller out there. Low and behold, it works perfectly! Yes, the flickering watermark will still be dimmed, but I think we can find a combination of colors or graphics that will help to make this visual leak less visible.

As of now, we're flickering it at a whopping 1 micros on the Arduino, but who knows what is the actual maximum update rate from Teensy 4.0 (maybe approx 30µs per LED strip + reset time).

One of the struggles I have with this project is coming with no clear concept. Of course there is no right or wrong on how to build on a project, I believe it is a cycle of concept building and tech/fabrication development, however, in my case, it is very much skewed towards the tech dev. I have more peace of mind if I try out first what technology will work and then match the concept towards it. So here are some variations that I've tried, all of them are still under the theme of the hidden watermark:

Variations

Rawfeeding the camera feed to the pixels. The variance of color is quiet unpredictable and hard to handle. Issues include:

- lighter color could reveal the watermark dimming easily. A possible solution is to put a background that's constantly busy to divert audience's vision from the watermark.

- the brighter (white) end of the pixel is rendered as green on the Neopixel, not sure if this is happening because of the way LED is being perceived by the camera. A possible solution is to constrain the color, and if it is too bright maybe just render it as white on the code?

Watermark + Camera [RGB Color]

This is a variation that I thought could solve some of the issues happening in the previous variation (watermark being obvious, color inaccuracy). Simply limit the colors to duotone, well, in this case it is monochromatic but technically it is a duo tone of white and black.

Watermark + Camera [Monochromatic]

I found that choosing really bright neon color combination will help reduce the visibility of the watermark in real life. Red and green are better combination compared to blue & green though.

Watermark + Camera [Duotone Blue/Green]

This is a new variation that I thought may be extending the concept further (if not straying away, i hope!). I got the idea because I notice the aliasing band that happens while documenting my work-in-progress. I thought, wouldn't it be meta if I refer to this aliasing as a way to make the subject (the LED lightbox artwork) 'copy' the camera artifacts on the environment and hinting the audience that this work has a little something to do with the camera shutter speed properties?

Watermark + Aliasing [Duotone]

Another extra iteration that I did is putting the camera stream underneath the aliasing, though I think it is a bit too much.

Watermark + Camera + Aliasing [Color RGB]

After talking to Gabe BC, he mentioned of Penelope Umbrico (in addition to Jim Campbell, who is mentioned by Danny Rozin during a feedback in class), and advises me that: if the work is about copyright, then maybe I could take a look into stock photos, AI-gen images, or images that are commonly used by people.

And so I tried to fetch AI-gen images from WIkimedia Commons. If I go with this variation, the work will definitely be talking about copyright and the debate whether AI-gen images can be claimed of its copyright. However, on the audience side, I'm not sure if this experience is enriching for them?

After doing all these iterations, I realized that my initial concept may not be just one actually. I thought I was interested in the copyright, but I think I might be more interested in the idea that using your mobile device (or accessible camera device) can alter the experience of one when looking into a photograph. And if we circle back to my initial inspiration -- which is to make fun of the people in art gallery where they take selfies without looking into the work for long, it seems that I need to put aside the copyright idea.

&TLDR;

All the above variations didn't make the cut due to my concern that the camera stream distracting from the watermark / secret message artifact. The ideas above may have arised due to my dilemma in finding a trick to compel audience to take a picture. I delved again, maybe a face filter could work, maybe instead of a watermark/copyright, the camera could be blocking the audience's face instead so they couldn't take their picture -- which defeats the purpose of the camera itself!

- snippet of video here -

That being said, the final iteration of this work for the 2025 ITP Winter Show fall between the options of:

1. Face filter + watermark -> more straightforward message towards audience. The work mocks you from taking a picture of yourself.

2. Aliasing -> more big brain energy, but presume too much of a knowledge from general audience.

3. Wikimedia Commons -> definitely copyright direction.

Documentation

Special thanks to Daniel Rozin, Sophie, Octavio, Christina, Billy Fadhila, Fabrizio, Cody Frost, and all friends :)