IndusBee 4.0
- Sensory Systems and Machine Learning for Optimized Bee-Tending and
Hive Keepers Assistance
Subject:
Due to remarkable work like the 'More than Honey'
documentary, it has entered public conciousness, that bees play a
crucial role in our ecosystem, but are subject to major detrimental
influences and threats from pesticides to parasites. This imposes an
increasingly hard to meet load and responsibility on hive keepers.
Numerous activities have been triggered in the last decade, trying to
improve bee monitoring and hive keeping by tools and techniques from
related disciplines, e.g., applying sensors and monitoring procedures
from general automation activities, home automation in particular. The
work presented here, targets on bringing knowledge and infrastructure
from the recent field of Industry 4.0
in conjunction with integration
technologies for sensors and electronics with embedded Machine Learning
and Optimization
techniques to exploit automation concepts for bee keeping, to
achieve efficient hive keeper assistance systems, and last not least to
minimize the required hive opening and intrusion by
sensor based monitoring. The investigated techniques will be
validated and extended in the bee hives of my BeeLab.
Abstract:
I was lucky, that from early childhood until the
end of my engineering education, bee keeping was always present in my
life and
triggered my curiosity on bees sensory capabilities and inherent swarm intelligence .
The contact with bees
provided me with improved understanding of nature and dependencies in
our ecosystem, though it also gave to me in my boyhood the painful
experience of monocular vision for nearly a week. The tending of bee
colonies throughout the seasons of the year was, and in part still is
very much based on the individual expertise, observation capability,
and the right decisions of hive keepers. The increasing
detrimental influences by environmental challenges and threats from
pesticides to parasites led to a dramatic increase in bee colony
mortality and impose an increasing load on hive-keepers. As an
engineer, the immediate question was and still is, whether tasks could
be automated and improved,
reducing the manual load of regular check-ups with the related opening
of the hives, as well as the expertise required by resorting to
resources and techniques from sensing, measurement systems, and from
Machine Learning to continously monitor and automatically assess bee
colony condition, and further support
the hive-keeper by the introduction of appropriate assistance systems.
The fields of
Ambient Intelligence, (Home) Automation, IoT, and
Industry 4.0 provide
the leverage by infrastructure and methods to be adopted to this
ecologically crucial domain. One subproblem in this field is the
analysis of
honey quality, e.g., from food fraudery activities related to honey
various sources of
contamination, blending etc. to
on-site quality assessment. This started to be addressed and is still
pursued in one our
related projects an
Lab-on-Spoon/Fork in
the context of our
E-Taster system as
mobile food inspection devices, also for honey quality:
Numerous engineers and hive keepers have been
attracted to that field of activity in the last decade and hive
instrumentation has seen significant attention and advance. Most
commonly, hive temperature and moisture is measured, logged, and
communicated to hive keepers mobile or related devices. Hive weight
monitoring has also been widespread and commercial solutions are
available. Bee activity has been deduced from weight change over the
day, but also by extra sensors, and additionally, the feeder level
module monitoring
has been added. More recent approaches from about 2012
onward until today extend the employed sensory palette to video and
audio sensors, e.g., cameras and microphones. In addition to
instrumentation and communication, intelligent monitoring and decision
making massively has entered the field. Examples of these activities
can be found, e.g., under
hive-monitoring
(with a focus on cheap
Arduino
based solutions,
Bee-Counter
etc.),
IoBee,
BeeandMe, or last not least the
OSBH project.
There are an increasing number of activities, publications, and also
patents. However, there is still a little headroom for extensions and
improvements related to adding new sensory capability, advanced
information processing capability, and employ advanced packaging
technology, e.g., from SiP (System-in-Package) domain for both
increased robustness as well as feasible prices of the required hive
instrumentation. Here ISE experiences, from sensor and electronics
integration technology to automated intelligent system design, in
particular,
from our Industry 4.0 related activities, e.g., in the
BMBF-project MoSeS-Pro or in
Polymer
Film Industry
process optimization, are going to be exploited. Currently, based on
low-cost Arduino & Raspberry PI platforms, a
modular hive
instrumentation and intelligent monitoring system is evolved, that
comprises the following features under cost and integration
constraints:
- Standard temperature and humidity sensing
- Standard hive weight measurement, simple extension to separate
honey storage measurement
- Standard feeder level monitoring
- Various visual monitoring approaches, e.g., with active
illumination beyond bee sensitity (above 530 nm) to avoid colony
disturbance and stress
- Distributed acoustical monitoring based on multiple MEMS
microphones for colony status classification from intrusion detection
to swarm indication
- Application of Image/Signal Processing and Machine Learning
techniques for robust automated evaluation of the data to keep hive
keeper interference to a just in-time minimum
On example of alleviating hive keepers task is related to the
required
routine inspection for the degree of Varoa mite investation of
the colony. In the currently dominant modular magazine hives with
open bottom, the colony debris including dead mites drops onto a
collecting 'dipher', that is manually inspected for each hive
for parasites, counting the number as an assessment figure of the hives
condition and treatment requirements. Common methods from Machine
Vision and automated quality control can be adopted to automate and
accelerate this process:
The various
pieces of debris, e.g., wax, propolis, bee limbs, and varoa mites can
be processed by image segmentation approach detecting candidate regions
for further classification (blobs shown in right picture). Then the
regions representing varoa mites can be automatically identified in the
next step, and
the mite regions can be counted. This assistance promises to provide
substantial speed-up for
owners of large hive numbers and additionally allows a detailed
archiving of each colony's results. Currently, a dedicated scanner
setup is pursued and a
more simple mobile phone App based variant is considered.
For
further visual remote hive monitoring, low-cost cameras for close-up
view with active illumination outside the bees perceptive range (>
530 nm), and even thermographic monitoring with moderate resolution can
be considered.In addition to remote interactive monitoring, Machine
Vision and Learning techniques for automated continuous monitoring and
focused alert generation for hive keepers can be considered:
In the next
steps of the work on IndusBee 4.0
the following
objectives wil be pursued:
- Extending
the instrumentation of bee hives by highly capable most recent MEMS
microphones and sophisticated Machine Learning (also examining
Deep-Learning vs. our proprietary automated intelligent system
design methodology) for advanced hive condition monitoring
- Merging of auditory and visual multisensor information in
on-line hive condition monitoring
- Investigating 'Electronic Nose' options, e.g., by amenable VOC
sensors, such as SGP30 or BME680, etc., for illness and other problems detection
- Appropriate wireless communication (WiFi, Lo-Ra,....) will be
regarded
- Proliferation of
ideas to the community as well as potential marketing of achieved
features/systems into product(s)
- Extension with self-x features of the hive monitoring equipment
for accurate as well as
dependable
sensory systems.
- Investigation of alternative/complementing 3D
integration/packaging (System-in-Package, SiP) technologies for
seamless integration & merging of every day life devices with
sensors & electronics. In particular, the in-hive sensors have to
be robust/immune to potential rough handling by the bees
- Harvesting/energy management, e.g., by adding standard solar,
thermal, or
other
harvesters.
Sponsoring
The project is entirely funded by private funds from Prof. Dr.-Ing.
Andreas König
|
Status: |
|
Running,
duration: 2017 - retirement and beyond
|
|
Partners: |
|
- |
|
Financing: |
|
Self-funded research |
|
PI/Contact: |
|
Prof. Dr.-Ing. Andreas
König |
|
Contributors: |
|
Prof.
Dr.-Ing. Andreas
König
|
|
Publications: |
|
|
|
|
|
|
|
|
|
A. König, Extending Bee Hive Health State Monitoring by Integrated Acoustical Sensing and Machine Learning, in Proc. 5th Int. Conf. on Sensors and Electronic Instrumentation Advances (SEIA 2019), Canary Islands (Tenerife), Spain, Sept. 25-27, 2019.
|
|
|
|
A. König, VarroaCounter - Towards Automating the Varroa Screening for Alleviated Bee Hive Treatment, in Proc. 5th Int. Conf. on Sensors and Electronic Instrumentation Advances (SEIA 2019), Canary Islands (Tenerife), Spain, Sept. 25-27, 2019.
|
|
|
|
|
Andreas König, IndusBee4.0 -- Integrated Intelligent Sensory Systems for Advanced Bee Hive Instrumentation and Hive Keeper's Assistance Systems, Sensors & Transducers Journal, 237.9-10, pp. 109-121, 2019
|
|
|
|
|
A. König. Integrated Intelligent Sensor Systems for In-Hive Varroa Infestation Control in Digital Bee Keeping. in Proc. of SMSI 2020, Vol. Sensors and Instrumentation, Chapter B3 Biosensors and Sensors for Biology, pp. 111--112, 2020. DOI 10.5162/SMSI2020/B3.1
|
|
|
|
|
A. König, BeE-Nose - An In-Hive Multi-Gas-Sensor Extension to the IndusBee4.0 System for Hive Air Quality Monitoring and Varroa Infestation Level Estimation, in "Advances in Signal Processing: Reviews", Book Series (S. Y. Yurish, Ed.), IFSA Publishing, Vol. 2, 2021, pp. 443-463. |
|
|
|
|
Andreas König, First Results of the BeE-Nose on Mid-Term Duration Hive Air Monitoring for Varroa Infestation Level Estimation, Sensors & Transducers Journal, 250.3, March, pp. 39-43 , 2021
|
|
|
|
|
A. König, Cognitive Integrated Sensor Systems for In-Hive Varroa Infestation Level Estimation based on Temperature-Modulated Gas Sensing, to be published in Sensor and Measurement SMSI Science International (SMSI), 2021.
|
|