Institute of

Cognitive Integrated Sensor Systems

Prof. Dr.-Ing. Andreas König

News

Staff

Teaching

Research

Publications

Conferences

Student Theses

Contact

Impressum

Home

The E-Taster Assistance System with Lab-on-Spoon and Lab-on Fork as 'Electronic Tongues'

         

Subject:

Daily life activities of food preparation can be challenging for consumers due to restricted skills and experience, loss of perceptive abilities (accident/aging) or due to product fraud and contaminations, commonly summarized under the term of food adulteration, which are exceeding human perceptive abilities in general. Thus, in our work the E-Taster-assistance-system for living assistance and food safety was conceived. Autonomous, multisensory, wireless measurement systems (Lab-on-Spoon/-Fork, LoX), which have been embodied as items of daily use, deliver as 'electronic tongues' sensory context for the required assistance. These integrated sensory systems can  be understood as Internet of Things (IoT) devices and they have implemented based on the Arduino platform, which has established itself as a common choice for IoT solutions. The E-Taster system includes  an electronic cookbook, that interacts with the multispectral LoX devices and also has special functions for sustainable use of food. For the rapid and dependable extension and adaptation to increasing tasks the DAICOX- System for the automated design of optimized technical cognition systems will be presented in the context of the E-Taster-system and Lab-on-Spoon application. The E-Taster project  offers both a solution for the underlying mass market at the end of the food chain and with the DAICOX system a general tool for the rapid design of intelligent IoT- and related systems. It was presented to public on CeBIT 2015, hall 9, booth D23 of Rhineland-Palatina and at SENSOR+TEST 2015, MAZet booth hall 12/386. Reports in the media have been in Rheinpfalz, 28th of April, 2015, in the SWR web and Landesschau Aktuell on Friday, 29th of May 2015.


Abstract:

Since ancient times the ones in power always had to fear the poisoning of their food to deprive them of power and life. Employing a taster to check every meal and dish before consumption was a common counter measure. These days, the problem of dangerous food due to contamination or adulteration has aggravated and broadened to the average consumer. Though there are common and quite numerous cases of unintentional contamination of of food,  an increasing amount of profit driven activities to extend food sales quantity by cheap and in many cases very dangerous additives can be found. Some examples of increasing danger for the consumer are, e.g., injection of water in meat or fish to increase weight,  mixing of salt or spice powders with  various cheap additives,  sweetening wine by anti-freezing agent (diethylen glycol), or the sale of recycled ot otherwise generated questionable oils as frying oil. an increasing number of these food falsifications or adulterations could be detected by contemporary sensory systems taking the role of the taster for the average consumer. In particular, integrated sensory systems allow the embodiment of such functionality in the shape of ordinary items of daily life, e.g., spoons, forks, bowls etc.
In our work we have conceived the E-Taster Assistance System, that employs devices such as, e.g.,  Lab-on-Spoon, to  assess  ingredients and also procesing step results in food preparation. Thus,  the E-Taster Assistance System can serve in Ambient-Assisted-Living-Scenarios to support users challenged by illness, age, or accident in assessing correctness, pureness, and freshness of ingredients and of food processing steps. Further, problems due to food adulteration, possibly out of human preceptive capability, can be detected and put to the attention of the consumer. The following figure shows the block diagram of our E-Taster Assistance System: 
 

The current stationary version of our E-Taster Assistance System bases on a smart-kitchen-server, which runs a system to communicate with the user, instrumented devices, a dedicated database, efficient pattern recognition. A key module is an electronic or e-cookbook with a recipe collection, that is controlled by gestures and employs speech output and possibly input. The e-cookbook collects sensor context from, e.g., Lab-on-Spoon, or other devices for the recipe steps. The Lab-on-Spoon has been conceived in prior work (CeBIT 2014) for liquid food analysis and  has been extended now to multispectral analysis capability. It bases on the popular Arduino platform, that is enjoying increasing acceptance for IoT device implementation. The following block diagram shows the architecture of the new multispectral Lab-on-Spoon, which will be presented to public at CeBIT 2015:

 

The multispectral Lab-on-Spoon (LoS-MS) is conceived as a 3.3 V system based on an Arduino Pro Mini (3.3V, 8 MHz). It offers  temperature sensing, multicolor sensing, and impedance sensing. This achieved by a UST Pt10k sensor and readout circuit,  a MAZet MCS6CS  multicolor sensor with two MCDC04 readout chips,  and AD5933 based impedance measurement in the range up to 100 kHz. The LoS-MS is equipped with active illumination and background light canceling. A white light LED and a 750 nm LED are employed in two cycles, as, e.g., the Center-Diode of the MCS6CS shows exploitable sensitivity for that wavelength. Communication with the server and the E-Taster Assistance System is achieved either by USB or  wireless by the XBee interface. The autonomous Los-MS includes an Li-Po-accumulator, that will be charged via the USB connector. As the employment of the MAZet MCDC04 was not possible due to the Repeated-Start-Protocol, the LoS-MS system has been implemented based on the SoftI2C-Library. A simple Arduino program for the MCS6CS by two MCDC04 will be provided HERE after CeBIT 2015. The following annotated photo shows a LoS-MS with an open package and indications the named components of the autonomous measurement system:

 

The e-cookbook of the E-Taster Assistance System requests the user at appropriate steps of food preparation to  fill the expected ingredient in the spoon and press the LoS-MS button for confirmation. This 'handshake' procedure prevents ambiguous situations, e.g., empty spoon or spoon filled by a mixture of residuals of previous ingredients. The requested sensory context of LoS-MS will be transfered to a pattern recognition module in the e-cookbook of the E-Taster system, processed, and a a decision will be made by a previously designed classification system. The user will be informed on the outcome of the test. The following picture shows the encapsulated autonomous  LoS-MS ready for measurement:

The e-cookbook of the E-Taster system is demonstrated in the following composite picture for the example of a simple Chinese dish,  called chicken Sung Bao, where the e-cookbook calls on sensor context at certain steps, exemplified here by steps, where soy sauce or vinegar shall be added to the sauce of the dish:


In addition to LoS-MS, a Lab-on-Fork (LoF-MS) for probing solid food, e.g., meat, cheese, etc. has been developed, which basically has the same sensing capability. The majpr difference is, that temperature and impedance are now associated with the fork's teeth, while the multicolor sensor was rotated by 90 degrees and changed the mode of operation from transmitted light or scattered light measurement to reflected light measurement. The following picture shows a LoF-MS USB-prototype before encapsulation:


The LoF-MS is demonstrated for one example of fish identification with three kinds of fish, i.e., tuna, salmon, and zander. The LoF-MS has been employed to measure three filets or chops from the named kind of fish, measuring both multi-channel color and impedance spectrum. The following picture shows the underlying measurement objects and the resulting feature space computed from both sensory channels, which shows clear discernment capability.  Each individual sensory channel showed for itself suitable discerment capability. Thus, authenticity of the expected fish kind, with potentially quite different price tag than possible substitutes, can be obviously be checked. Freshness grading of the fish will be next in the LoF-MS investigations. Extensions to meat, cheese, etc.  will also be regarded.


The scope of the considerations have further been extended by LoF-MS to the investigation of powders, such as spice powders, coffee, tea, salt etc. A common problem in food adulteration is, that more or less dangerous, cheap additives are added to the original ingredient to increase sales profit. The following figure shows impedance magnitude data for the common example of  adding chalk powder to salt for this purpose, where the three clusters indicate pure salt, pure chalk powder, and a 1:1 mixture:


The E-Taster system and the 'electronic tongues' Lab-on-Spoon and Lab-on-Fork in their recent multispectral and autonomous version are presented to public at CeBIT 2015 in hall 9, booth D23 of Rhineland-Palatina. The following pictures from the CeBIT booth show the visit of the minister of the MSAGD ministry of Rhineland-Palatina, Mrs. Sabine Bätzing-Lichtenthäler, and the visit of the secretary of state of the MBWWK ministry for science and technology of Rhineland-Palatina, Prof. Dr. Thomas Deufel, where the e-cookbook application of the Lab-on-Spoon and contamination detection in powders, e.g., salt, by Lab-on-Fork, were presented as life demos:

    

    

In the wake of the CeBIT 2015, the E-Taster assistance system and Lab-on-Spoon/Fork was reported on in Rheinpfalz, 28th of April, 2015, in the SWR web and Landesschau Aktuell on Friday, 29th of May 2015.

In the next steps of the work on E-Taster assistance system and Lab-on-Spoon/Fork the following objectives wil be pursued:


Sponsoring

The major color sensor manufacturer MAZeT has been attracted as sponsor to the Lab-on-Spoon project and supports us, in particular, with the most recent multispectral MMCS6CS sensor for our current multispectral Lab-on-Spoon version.


                          

                          


  Status:   Running, duration 04/01/2014 - today
  Partners:   -
  Financing:   Self-funded research
  PI/Contact:   Prof. Dr.-Ing. Andreas König
  Contributors:   Prof. Dr.-Ing. Andreas König (Conception of the general E-Taster architecture, Lab-on-Spoon, and Lab-on-Fork and their phyiscal implementation, applications, and measurements.)
M.Sc. Kittikhun Thongpull (Autonomous Lab-on-Spoon/Fork functionality implementation, intelligent system design with sensor fusion, dimensionality reduction, hierarchical classification, and anomaly detection, E-Taster integration, and applications),
Simon Hook, Marius Gräfe, Johannes Bayer, and Simon Böhm (E-cookbook module with Kinect-based gesture control and Lab-on-Spoon interface for sensor context acquisition during cooking process.),
M.Sc.Abhaya Chandra Kammara (Kinect-based gesture control development for e-cookbook and other related support.)

Technical support:  The packages of the autonomous LoX devices have been designed and printed by F. Witthaus, TU central workshop, and the layout of the AFE board has been designed by M. Schultheis.

  Publications:  
     
A. König, "Integrierte Intelligente Sensorsysteme - Vom Labortisch ins Anwendungsfeld", GIT Labor-Fachzeitschrift, Schwerpunkt Laborautomation, GIT Verlag, 59. Jahrgang, 2/2015, S. 37-40.
      A. König, K. Thongpull, "Lab-on-Spoon - Vorkoster mit elektronischer Zunge", Elektronik Messen+Testen, WEKA-Fachmedien, 5/2015, S. 46-49 .
     
K. Thongpull, D. Groben, A. König, "A Design Automation Approach for Task-Specific Intelligent Multi-Sensory Systems - Lab-on-Spoon in Food Applications"  tm - Technisches Messen. Volume 82, Issue 4, Pages 196–208, ISSN (Online) 2196-7113, ISSN (Print) 0171-8096, DOI: 10.1515/teme-2014-0009, April 2015