DishSnitch: The Who Left Dirty Dishes in the Sink" Detector"

Dirty dishes! Our team is dealing with an abundance of those. Argh!

Which brought our researchers to develop a DishSnitch. Yeah, an automated app that detects and shames the culprit by sending the full evidence—including pics—to the company’s dedicated Slack. **Face palm**

For those who are interested in the DishSnitch, we placed the app’s files free to grab on Github—you’ll find the link at the end of the post.

Introduction

This project was the culmination of an exciting two-day hackathon at Panorays, Panackathon. The idea was that teams worked on a dedicated project related to the company, under the theme of automation.

Considering that the dishes take up nerves, time and, well, smell bad, for the benefit of the company, we just had to create this DishSnitch. Together with me, this app was developed by Stephan Gross and Yarden Zemach, members of the Security team. (Necessary pat-on-the-back note: We actually developed two separate unrelated automation projects during these two days. The other project we worked on will be integrated within the Panorays platform shortly. Exciting times!)

The Non-Solution

To address the dirty dishes problem, a sign was first placed above the sink as a reminder for people to wash their dishes.

Did this achieve the desired results?

The Actual Solution

But no worries, because now we have DishSnitch! This handy gadget was created to identify the culprits who left their dirty dishes in the sink and send their pics together with the evidence of the soiled dishes in the sink to the company’s dedicated Slack channel. In parallel, a recording calls out to the culprit in a loud, booming and warning voice. Finally, plastic missiles are shot at the culprit.

DishSnitch does this through image recognition of both the person and the sink, alerting through Slack’s API.

The DishSnitch Implementation

We connected a laptop to:
1. Two video cameras installed near the sink. The first faces forward to record the person that approaches the sink, and the second faces downward to monitor the status of the sink.
2. A speaker
3. A plastic missile USB launcher
We created a database of employee pics on MongoDB, so that when a new face is detected, it connects to that MongoDB instance and matches against it.
We then used OpenCV, an open source computer vision and machine learning software to detect both dishes in the sink and the individual approaching the sink. The OpenCV is configured to be in an initial state of “clean.” Once it detects dishes in the sink, its flag is turned to “on.” When an individual approaches the sink, the camera takes a picture and OpenCV detects that individual. Since that individual may simply be cleaning the dishes, an additional checkup of the sink state is performed when the individual leaves.
If OpenCV detects that indeed there are dirty dishes in the sink when the individual leaves, it sends a predefined shaming message to the company’s dedicated Hall of Shame Slack channel. This was performed by integrating OpenCV with Slack’s API, where the message contained the name, image of the employee and the dirty dishes.