Lab-on-Spoon - Multi-Sensorial 3D-integrated Measurement System for Smart-Kitchen and AAL Applications
Subject: (see our most recent extension of this research, the E-Taster Assistance System, also at
CeBIT 2015, hall 9, Booth D23 ! )
This research combines the increasingly popular, versatile, and powerful technique of Impedance Spectroscopy with our activities on smart-environment, e.g., in our Smart-Kitchen scenario. In this food processing or culinary scenario, as well as food safety monitoring devices related search, we pursue the creation of assistance systems for the tasks of food storage, quantity and quality assessment, as well as preparation result assessment. One key aspect is the providance of capable sensory context integrated by todays' powerful available System-in-Package (SiP) integration technologies from the More-than-Moore direction of the ITRS roadmap in devices of daily living. The first item under investigation was the design of multi-sensor system embedded in or better embodied by a spoon. Pioneering work on smart-spoons and related devices has been done in the last decade, e.g., by MIT (Counter Intelligence project, Dr. T Selker et al.). The focus in our work is on achieving a highly capable analysis system as a component of an assistance system in the form of an multi-sensor, autonomous, wireless, low-power device in the shape of everyday life device with a focus on cheap mass-market realization. From 3D-printing technology to 3D electronic packaging technologies are under investigation for this aim. In the first prototype, an embedded realization of Impedance Spectroscopy is combined with color and temperature registration and applied jointly with Computational Intelligence methods to tasks, as e.g., oil state classification. See a short video presentation of Lab-on-Spoon by EbruTV Technology Feature Update, No. 45 (Folge 45).
Abstract:
In the last decade, the research on miniturized, distributed,
autonomous networked sensor systems has been pursued intensively and
found numerous applications. The vision of Smart Dust from Berkeley has
been one of the driving factors in the development. In
particular, distributed intelligent systems in smart-environment,
ambient intelligence, and ambient assisted living (AAL) have
significantly benefitted from the advance and new application fields
have merged, e.g., Industry4.0 (Smart-factories), Internet-of-Things
(IoT), or Cyber-Physical-Systems (CPS). From the early conceptions to
most of the state-of-the-art systems, three potentials for improvement
can be indentified. Firstly, the current realizations of sensing could
benefit a lot from the know-how and standards in sensor, measurement,
and instrumentation community, both in accuracy and diversity of
methods. Secondly, the robustness of systems and the capability to
correct and maintain themselves, commonly denoted as self-x- or
self-*-capability,
is required and can be adopted from related activities in the community
of adaptive and dynamically reconfigurable hardware. Last not least,
the third issues is the exploitation of current MEMS and advanced
packaging technologies, e.g., 3D-printing or 3D-electronics and systems
packaging.
For this aim, the smart-kitchen and related AAL scenario has been chosen to employ, combine, and refine technologies from the three lines of research mentioned above to achieve more able, more capable, and unobtrusive systems for assisted working and living. The topic of smart kitchens has been pioneered by MIT (Counter Intelligence project, Dr. T. Selker) and has seen numerous follow-ups in the last decade. Also, the scenario of a smart spoon has been tackled the same MIT group with remarkable sensing capability, e.g. temperature, conductivity (DC), pH-value. New lines and capabilities come from advanced sensor and measurement approaches and their efficient integration. Impedance Spectroscopy and its translation to embedded or Integrated Impedance Spectroscopy (IIS) is one of the pursued new research directions, e.g., for a Lab-on-Spoon system to provide more powerful sensorial context in Smart-Kitchen and related scenarios. The following picture shows the Kinect depth and color image based gesture controlled central unit of the ISE-Smart-Kitchen, which includes a prototype of an electronic cookbook:
The next picture shows a close-up of the opened interactive electronic cookbook for one particular recipe:
The Lab-on-Spoon is conceived to provide for the context of each preparatory step the sensorial information or feedback on food ingredient quantity and quality, i.e., classify both the correctness of the kind of ingredient as well as its quality or state, in particular, judging the freshness to avoid rotten ingredient inclusion in the preparatory process. Also, the robust detection of contaminations can be identified both as relevant for immediate application and additional research. Thus, users potentially impaired in their qualitative and quantitative judgement on ingredients and preparations process step results' by either restricted skills and experience or by loss of perceptive capabilities due to accident or aging can be supported by our system. Lab-on-Spoon represents a front-end component of a corresponding assistance system in the ISE-Smart-Kitchen, which provides sensorial context to restore or improve users' perceptive abilities. The sensor and measurement part of the Lab-on-Spoon system is illustrated in the following picture:
Further investigations
and measurements for larger
data sets and new applications are in progress. The next one, with
regard to CeBIT 2014 partner country UK, is the
distinguishment of teas represented by six different types of
tea, i.e., Ceylon, Earl Grey, Darjeeling, Jasmine, Peppermint, and
Camomile, which are subject to measurement of color and impedance
values by
the Lab-on-Spoon prototype with settings as given
above for the previous examples. Additionally, the relevant features
for the task have been selected from the color and impedance
values by automated feature selection:
In the case of the teas a
slight confusion between
Ceylon and Earl Grey can be observed for the current hardware,
which will be tackled by the sensory enhancements in progress.
The next example of distinguishment of beverages regards four different
types of
beer, which are subject to measurement of color and impedance values by
the Lab-on-Spoon prototype with settings as given
above for the previous examples. Again, the relevant features
for the task have been analysed from the color and impedance
values by automated feature selection, which also help to significantly
reduce measurement time in future measurements for this task:
The next and key one is
typical to Rhineland-Palatina
as a
major wine-producing state and deals with the
distinguishment of seven different types of wine which are again
subject to measurement of color and impedance values by the
Lab-on-Spoon with settings s given
above for the previous examples. Investigations of the relevant
features
for the task has been conducted from the color and impedance
values by automated feature selection as in the previous cases:
This application is quite
complex and the current
state of the hardware
implementation, as for the tea tasting example, meets limitations in
discerment, which we
confident to overcome by our sensory extensions on the way.
But the example has another interesting application, as the case of
wine spoiled due to time and storage conditions or due to contamination
gives rise to another challenge for Lab-on-Spoon.
We let ourselves be inspired by a problem wine industry faced about two
decades back, where wine was
contaminated by Diethylenglycol to sweeten it. In our experiment, also
employed as CeBIT demonstrator,
we use a much more harmless substance glycerol, which also serves as an
anti freezing agent.
About 10% are added to dry white wine for the Lab-on-Spoon system to
detect. The sensory data for pure and
contaminated wine are recorded as in the previous cases. A plot of the
feature space is given in the following:
The classification was achieved by training a suitable Support-Vector-Machine on normalized and selected measurement data in Orange. Contamination detection capability of Lab-on-Spoon could be proven in numerous life demos on CeBIT, in particular, on the occasion of a visit of the prime minister of Rhineland-Palatina, Mrs. M. Dreyer, to our booth.
The Lab-on-Spoon prototype extension to pH-value and viscosity is under investigation. The temperature, color, and Impedance Spectroscopy based prototypes in tentative wired and XBEE-wireless version will be on display on CeBiT 2014, exhibition hall 9, booth D23 of Rhineland-Palatina (Rheinland-Pfalz), presenter M.Sc. Kittikhun Thongpull of ISE. The first proof-of-principle demonstrator of Lab-on-Spoon with temperature (UST pt10k), color (MAZeT MCS3AS), and impedance spectrum measurement (AD5933 with AFE) and Arduino pro mini based version with active LED illumination is given in its unsealed USB-variant here:
The following picture
shows a sealed, wireless, and
autonomous Lab-on-Spoon as demonstrated at CeBIT.
It has one more functional extension of value for life assistance. The
temperature of the spoon content is continuously monitored and
translated into a color of the spoon illumination. Low temperatures can
be coded by blue, very high temperature by red color with a seamless
transition of color shade in the programmable intervals. This can be
exploited as a heat warning to avoid hurting by too hot spoon contents,
but is can also application-specifically be changed by the smart
kitchen host, i.e., be programmed to the interval of water temperature
required for the
quite sensitive yeast bactery cultivation needed for, e.g., bread
baking:
The following pictures from the CeBIT booth show the visit of the prime minister of Rhineland-Palatina, Mrs. M. Dreyer, where the contaminated wine detection was presented as life demo:
Status: | Running, duration 11/01/2011 - today | |||
Partners: | - | |||
Financing: | Self-funded research | |||
PI/Contact: | Prof. Dr.-Ing. Andreas König | |||
Contributors: | Prof.
Dr.-Ing. Andreas
König (Concept, multi-sensor Arduino Lab-on-Spoon design and
implementation, measurements, and CI on
QuickCog, CeBIT 2014 presentation), M.Sc. Kittikhun Thongpull (Python drivers, CI on Orange, CeBIT 2014 presentation), M.Sc. Abhay Chandra Kammara (Smart-kitchen, e-cookbook and sensor context) Students: Thomas Gräf, Color sensor module design, student assistantship (HIWI) funded by AmI research center in 2004, reuse for Lab-on-Spoon. Thomas Bölke, Diplomarbeit, MPT, reuse of some Arduino code from ISE DeCaDrive-project. A. Renner, L. Minghan (Semester projects (Studienarbeiten), design of new Impedance Spectroscopy CMOS chip) Students (Energy-Micro-based activity): Z. Espejo (Master thesis, Energy Micro uC, breadboarding, end March 2013), D. Los Arcos (Master thesis + student assistantship (HIWI), Energy Micro uC PCBs, measurements, 3D-prints, end Jul. 2013), H. Pekmezci (Semester projects + student assistantship (HIWI), Python driver for Energy Micro uC, guided measurements with basic CI, end Dec. 2013), (Please note, that the Energy Micro based activities have been suspended and replaced by a more reliable and capable Arduino/Orange based multi-sensor Lab-on-Spoon-system) |
|||
Publications: | ||||
A. König, K. Thongpull, ”Lab-on-Spoon – a 3-D integrated hand-held multi-sensor system for low-cost food quality, safety, and processing monitoring in assisted-living systems”, Journal of Sensors and Sensor Systems, 4, 63-75, doi:10.5194/jsss-4-63-2015, 2015 | ||||
|
||||
König, A.: Design and Application of Intelligent Integrated Impedance Spectroscopy Systems with Self-x Properties. In Book of Abstracts: Workshop Gesundheitszentrum Palatina, TU Kaiserslautern, 4. Jul., Kaiserslautern, 2013 | ||||
|
||||
Pekmezci, H., Los Arcos, D., König, A.: Embedded Impedance
Spectroscopy for Lab-on-Spoon Realization in Living Assistance Systems
and Intelligent Environments. In Abstract Book: International Workshop
on Impedance Spectroscopy, IWIS 2013, (Student
paper) pp. 41-42, Chemnitz, 25-27 Sept.,
2013.
Andreas König, “Automated and Holistic
Design of Intelligent and Distributed Integrated Sensor Systems with
Self-x Properties for Applications in Vision, Robotics, Smart
Environments, and Culinary Assistance Systems.” Invited Talk, Int.
Conf. on Neural Information Processing of the
Asia-Pacific Neural Network Assembly (ICONIP'08), Book of
Abstracts, pp. 69-70, November 25-28,
Auckland, New Zealand, 2008
|