OptoSensor
OptoSensor is an open source prototyping platform based on the principal of discrete distributed sensor arrays to create an low resolution image sensor to be utilised for multi-touch purposes. The project was developed as part of my university thesis, it was designed to be a low cost thin form-factor methodology to implement optical sensing in conjunction with a regular an off the shelf liquid crystal displays. OptoSensor technology is based of both Microsoft’s Thin Sight Technology and Inspire AG/Swiss Federal Institute of Technology Zurich FLATIR technology.
This project was implemented using a combination of existing sensing technologies to create sensor that is more robust and open than those available currently in the market. By using a relatively large array of IR sensors totaling of 128 sensors on a prototype board behind a traditional LCD panel and an IR light source in front of the panel, we are able to augment the display with the ability to sense a variety of objects near or on the surface; including finger tips and hands, and thus permitting us to enable multi-touch interaction. By creating a low cost high fidelity image sensor we are able to take advantage of optical sensing which also allows other physical items to be detected, and thus permitting interactions using various multi-modal interaction schemas.
The current prototype module design consists of 7″, 16:9 ratio, 128 sensor, board which is controlled via a dedicated microcontroller unit (MCU); the design is implemented in such a way that if needed multiple modules can be connected together (with appropriate modifications to both the firmware and hardware). The current hardware was designed to be flexible and easy to use as such the MCU chosen was based on the Arduino platform. The module utilises an Arduino Mega development board which is based on ATmega1280 MCU, as such the firmware can be easily modified by anyone using the free Arduino IDE.
The boards can be built by hand or be purchased as kits (Future), the firmware is open and can be downloaded for free. The reference designs CAD files are available under an open source license you are free to adapt them to your needs.
Getting Started
To get started, follow the instructions for your operating system: Windows, Mac OS X or Linux; If you’re having trouble, don’t hesitate to ask questions.
Hardware
Information about the OptoSensor hardware, including reference designs.
Learning
Examples software of how to work with OptoSensor.
Reference
Reference functions for the MCU firmware.
Interactive IO & More 
Hello Taha.
I am Brent from Australia. I’m very interested in building a thin MTT and am wanting to start ASAP. Your project here has really caught my attention and I would really love to build it. I have just finished my first year of Electronic and Electrical Engineering and this sort of project is right up my alley.
Being early on in my degree means that my knowledge is somewhat low however I am an Electrician by trade and have the practical know-how. My question is, do you have any detailed plans on how to build what you have done here? I am wanting to do 4 modules forming a rectangular shape to cover my 22″ LCD monitor.
I have downloaded your BoardLayout/Schematic but frankly am a little confused about exactly what components I need and how many etc.
Before starting the actual construction I want to map out my exactly layout and all components etc with a detailed plan and list of expected costs etc.
Any help at all would be appreciated.
Hi Brent what exactly are you confused about? I’ve got a list of components on the hardware page, if you are looking for the number of components used you can look at the schematic it says the number in there. As a quick overview there are 128 photodiodes, 128 smd resistors, 16 ADCs (you can cut this by using a 16 MUXs linking the output to the 2 ADCs, but this will increase complexity and slow the system down), 16 ceramic capacitors. this for one MCU. Now if you are converting to be used on 22″ monitor then you need to look at cascading systems… where this child board will be connected to a Master that controls all child boards, you can do this by using SPI, I2C or even LAN.
Hope this helps.
Hi Taha,
This is Michael Shing at Singapore. I am interested in build a replica of your design and perhaps build a bigger touch device later. My understanding for software aspects are as follows:
– No firmware required
– You have written a test application on Java to test the device.
Is this correct?
Thanks, Michael
There is a firmware required… since the project is based on the arduino development board, there is a firmware you have to download on to the arduino.
Yes using java i wrote 2 test applications both are very basic. This project was meant to be for educational research so as such no real work applications were made for it.
I see, thanks, Taha.
By the way, is there any particular requirements / arranagements for the LCD and LED panel used? How would they work the best based on your experience?
And is it possible to let me have the test applications? So, I don’t need to re-write something similar again. I would first test the setup before writing any more meaningful applications.
Thanks, Michael.
Sorry for the delay in replying… well yes and no you need to make certain considerations such as what happens to light after it passes the diffuser or the fernel lens or what happens after removing the reflecting paper of the backlight…. you need to compensate for those things.
Hi Taha,
You spoke about the LED backlight on the video. I understand the ‘whiter’ the backlight it is, the better the light one would get on the LCD display. However, there is no white LED. What did you use? Any problem with actual color display?
In addition, I tried to figure out where did you place the backlight in relation to the sensor board. Could you throw some light on this.
Much thanks,
Michael.
The monitor i chose had CCFL Lights that used to shine inside of an acrylic wave guide i replaced those CCFL lights with White ultra bright LEDs. If you use a LED display you wont have to do this i believe.
Hi Taha,
Could you also let me know what were the LED backlight that you used.
Thanks again,
Michael.
i just bought bright white led ribbon cable from ebay
Hi Taha,
1. On this site, it mentions that the reference designs CAD files are available under an open source license. Could you point me to that.
2. In term of the prototype physical layers, I saw the following:
i. LED ligh plan in front of the LCD;
ii. LCD panel (inclusive of the LED backlight);
iii. Sensor board attached to the back of the LCD;
iv. Signals are then fed into the Arduino kit for pre-processing;
v. Processing siganls are then fed into laptop via USB interface.
Is that corrrect? If this is the case, I do not understand how could the sensor board work behind the LCD without light passing to it.
Thanks,
Michael.
You can find the links in the getting started page… you are correct with your understanding. the light reflects of the tip of the input device and is reflected back into the sensor array. IR is pretty much permeable through an LCD panel provided you don’t have a high ppi.
Hi Taha,
Could I engage you as a consultant to build a larger size prototype?
To anyone out there,
If and only if Taha is not interested and you have done so successfully, please drop me a mail (mikeshing@gmail.com) for further discussion.
Cheers,
Michael.