This article was Part 4 a four-part series on multi-machine agricultural systems, addressing the integration of fleet, task, and mission management in this area.
Fleet and task management systems have developed along independent or loosely coupled paths. Multi-machine systems require both the integration of these and the addition of mission management, which considers the overall workflow of all machines in the system. A significant challenge with multi-machine systems is to consider how this can be done effectively to result in an intuitive workflow for the user, while also leveraging existing systems, integrating with industry standards and common practices, and minimizing support infrastructure. This requires an understanding of cloud systems, IoT (Internet of Things) technologies, and the state and evolution of the agricultural industry for connected systems and is a major component of effective multi-machine systems.
Cobbled Together Solutions
Part 1 of this series provided an overview of the split in the agricultural market between fleet management (machine location and health) and task management (job-related control and monitoring) systems. These task and fleet systems have developed independently in the agricultural machine industry for a variety of reasons explained in Part 1, but are increasingly needed to be brought together. Efforts to do this so far have mostly resulted in a clunky user experience due to challenges in limited access to key data, independent telematics/cloud systems, and independent manufacturers with closed systems. Progress has been made in many of these areas over the last few years, and has resulted in some realizations:
- Closed systems are a losing proposition – Early on in the IoT revolution in agriculture, it looked like several OEMs, as well as technology providers, had been trying to develop connected system platforms that would be a closed system, working with their equipment (or authorized partners), and blocking out all others. The temptation for the large players in the market is that they can dominate the market and provide a cleaner user experience for people who buy into the system completely. However, while this model may work for Apple, it has become clear that this is not sustainable in the agricultural industry, due to diverse OEMs, farm management systems, and technology providers, and farmer demands for more options. This was a lesson learned 30 years ago for electronics in agriculture, and spurred the development of collaboration efforts such as ISOBUS. The needs variation is so diverse in agriculture that no one company can effectively meet them all. Most companies have now realized this, and it has kicked-off a new atmosphere of collaboration in determining how systems can work together.
- There will not be a single provider of “THE” cloud solution – Individual organizations had also made early attempts to become THE cloud provider for all (or a large part of the market). This is a variation of the closed system, where only the data management is centralized, allowing for many different machine telematics systems to connect, as well as farm and fleet management systems. The value proposition to partners with this approach is that the overhead of the data management is handled, and each partner only needs to manage the system interface. This central data control has not been successful, as many organizations want to remain in control of how their customers’ data is managed for their application. It is clear at this point that there will be many cloud systems that need to share data for a cohesive user experience.
- Cloud-to-cloud systems are the future – What has developed is a cloud-of-clouds environment, where it is clear that there is strong value in different cloud systems existing for specific purposes, but these do not prevent interaction with other systems. This has a huge advantage of allowing OEMs to develop cloud-based multi-machine and data management systems specific to their needs, without having to be concerned about the system needing to be everything to everyone. This is made possible by the maturing of technology in this area, which lowers the barriers to creating and maintaining a cloud system that could be specific even to a single application, without worrying about double-work efforts. A large OEM may even have multiple cloud systems for different equipment types that can each meet the needs of those specific applications, but still share data, have integrated user authentication, and appear seamless to the end-user. This has really freed OEMs up to focus on their near-term applications without having to build the perfect cloud system to handle everything in the future, and allowed them to shape their customers’ user experiences to best fit the need. The DataConnect effort started with John Deere, CLAAS, 365FarmNet, and CNH; the ATLAS effort by AEF; and the Agrirouter by DKE-Data are all different (and potentially compatible) approaches that are in line with this cloud-to-cloud approach, giving strong signaling to the industry that this is the future.
These realizations provide a direction to the industry, but still are not providing strong user experiences that integrate all aspects of fleet and task management. What it has done is provide “permission” for machine manufacturers of all sizes to be confident in cloud-based system development that meets their specific needs, without worry of being shut out of the industry from a compatibility perspective. This allows OEMs to develop systems that are focused on the workflow and integrate all aspects of fleet and task management in a way that suits the application.
Autonomous Mission Management
While task management and fleet management are on a path where they may start to come together slowly for operator-driven systems, this is something that will have to be reckoned with sooner in autonomous systems.
A divided workflow between fleet and mission management for autonomous machines will result in a user experience that is so difficult, it will prevent adoption. This becomes clear when applying the process described in Part 2 of this series, and walking through an autonomous machine workflow from the user’s perspective. JCA has walked through this many times for different applications, shaping our Autonomous Framework to meet the needs arising from common problems.
A mission for an autonomous machine consists of both the path (where the machine will travel), and the task (what the machine will do). The many different problems can be categorized broadly into two types:
- Coverage problems – Coverage-type problems are common in agriculture for seeding, planting, spraying, tilling, and harvesting, where a machine needs to cover an entire area during its application. These problems have been the primary focus of task management technologies in precision agriculture applications. Common functions include planning prescriptions, monitoring as-applied maps for planting/seeding/spraying, and yielding with harvesting, as well as applying dynamic control functions such as variable rate application, section or overlap control, and auto-steer. This mission type can have a heavy pre-planning stage, where the path and the task goals can be determined before mission execution, with relatively small amounts of dynamic changes to the plan needed.
- Dynamic mission problems – This category lumps together “everything else” that does not fit into coverage problems. This is a wide range of other applications that require much more dynamic adjustment during operation, and minimal pre-execution planning is required, with more planning on-the-fly based on conditions in the field during execution. Examples could be baling, bale collecting, grain handling, seed refilling, refueling, rock picking, grading, and many other applications.
In reality, problems can also fit somewhere between these two types. For example, row-following problems are a variation of coverage problems, but with a high amount of dynamic planning needed. This is found in orchards, vineyards, corn fields, and other situations where an area needs to be covered, but there are explicit constraints on machine direction and alignment to the covered area.
An autonomous workflow generally consists of 5 stages: mission planning, mission deployment, mission execution, mission monitoring, and mission analysis. These are not necessarily sequential, and may be occurring simultaneously, depending on the application. The implication on task and fleet management for each of these shows the necessity for integration.
Planning of any autonomous problem will involve defining work areas (geofences) where a machine can operate safely. This can consist of multiple types of areas, such as:
- Task areas – areas where the machine can perform its core task
- Travel areas – areas where the machine can travel, but not perform its task (like a path between task areas)
- Keep-out areas / obstacles – areas where the machine cannot go within the overall work area
Setting of geofences is a common function of traditional fleet management systems, typically providing notifications when machines leave particular areas. For autonomous machines, further functionality is needed as it relates to the operation of the task.
Planning for coverage problems may also involve prescription planning, where the specific goals of the task are associated to geographic areas. This is a common function from traditional task management systems, where a specific application rate is defined for areas, based on agronomic analysis, to optimize inputs for production. Often, this prescription plan is generated from a farm management system that has advanced analytics used for generating a prescription based on previous production results, soil characteristics, drainage, and other environmental characteristics.
The machine path for coverage problems also can be generated in the planning stage, where the size, dynamics, and characteristics of the machine are considered to develop a path to drive for the coverage problem.
The planning for dynamic problem applications will consist of the work area definition, but either minimal, or more customized application-specific path and task planning at a pre-execution stage. The planning for these applications often involves a path generated upon execution from the current machine location to a target (may be a moving target), and can require considerations of approach, and heading to the target.
This planning stage includes elements that exist in traditional fleet and task management, and (especially for coverage problems) there is a strong benefit for using defined formats (such as those defined in ISO11783 for Task Controller data) for integration with farm management systems. However, to be useful to the operator, the workflow cannot separate these items, but rather must guide the user through a simple method that is tailored to the needs of the application.
Deployment of an autonomous mission involves sending a plan to a machine in preparation of execution. This deployment often consists of transfer from a cloud system to the machines, and may be done prior to execution, or dynamically, for multi-machine dispatched systems. Deployment mechanisms again exist for both traditional fleet and task management systems, but these require additional safeguards for autonomous systems, as well as a simplified workflow for users.
Mission execution is the core of the autonomous mission. Here, the machine needs to follow the defined path determined in planning (either pre-execution or on-the-fly), and execute its task, while adjusting to the environment and conditions with perception systems, safety and alarm conditions, and through guidance and navigation systems. Task controller (e.g., variable rate and section control) and auto-steer capabilities that have been developed in precision agriculture systems have basic elements of what is needed for autonomous machine execution of the task, and fleet management systems also have basic elements of machine health and safety systems. However, these are just a small slice of the scope of what is needed for autonomous systems, and these also now need to be integrated into one system – implement and drivetrain, and with much more capabilities than are needed with operator-driven systems.
Multi-machine systems complicate this further with communication between machines to coordinate functions, task-based coordination and dispatching of machines to execute an overall machine mission. Autonomous machine execution is often viewed as an evolution of precision agriculture technologies, when in reality, this is not an incremental evolution, but a major jump in complexity. Trying to build autonomous machine execution on existing technologies designed for task and fleet management in operator-driven systems is like bringing a knife to a gunfight. A whole set of new technologies are required to manage these systems, combining technologies developed from industries that have integrated complex systems (robotics, autonomous on-highway vehicles) with proven technologies in agricultural machine controls.
Operators in the work area who are in charge of the machines need to be able to monitor an autonomous mission during execution, and remote users need to be able to see mission status. Monitoring includes viewing task and path progress, as well as observing functions that have been part of traditional fleet management systems, such as machine locations, machine health and status, as well as alarm monitoring.
Traditional task monitoring for operator-driven systems has mostly been confined to in-cab solutions, and standards relating to this have been developed with the ISOBUS Universal Terminal (UT) function. While the UT has proven to be a success for operator-driven systems in agriculture, it does not extend well to autonomous machines. Here again, effective scalable autonomous machine solutions need to integrate the task and fleet management functions into a combined workflow that does not consider these as separate systems, but rather integrates elements of both systems. Furthermore, task monitoring for multi-machine autonomous systems means not looking at the operation of only one machine, but the collective machine task progress for the whole mission. This is a new paradigm for task monitoring that is no longer a one-to-one mapping of operator to machine – one that was never envisioned for operator-driven system technologies.
Mission execution produces a wide range of valuable data that can be analyzed for improving operational efficiency and agronomic production. This includes data often analyzed as part of fleet management systems (fuel usage, alarms and health status, distance travelled, operating time), as well as task management analysis related to the specific application (e.g., yield, coverage, inputs used). Here is where the integration back to farm management information systems (FMIS) is valuable, as the value of the operational and agronomic data extends beyond only the machine function. The cloud-to-cloud systems with defined APIs and standardized task data formats enable integration with various FMIS to provide advanced analytics with mission data.
Evolution of Task and Fleet Management in Multi-Machine Systems
Some of the functions that exist as part of traditional task and fleet management solutions are needed within an autonomous machine workflow, but because these need to be integrated, cobbled together solutions will not work. These must be customized for each specific application with an integrated workflow. The need for integration with farm management systems at the planning and analysis phases shows that technologies and standards (such as ISOBUS Task Controller) for shared data formats should be built within the current trends in the industry, moving towards data shared between cloud-to-cloud systems with open APIs. On the other hand, the need for customization to fit the requirements of the autonomous application across multiple machines likely means moving away from technologies intended for one-to-one operator to machine interfacing, such as the ISOBUS Universal Terminal (UT).
Customizing the entire workflow to meet the needs of the application is critical for multi-machine autonomous systems, blurring the differentiation between fleet and task management into integrated mission management. This trend is also enabled for operator-driven multi-machine systems through mobile device technology and cloud systems that can share data at the cloud-to-cloud interface level.
Bringing It All Together
Through this series on multi-machine systems, we saw that connected systems and automation technologies are changing the landscape of agricultural machines. Machine manufacturers that want to lead in this new environment need to shift their thinking from the function of a single machine to the entire workflow of the agricultural task. Applying this thinking can help industry innovators realize which existing technologies should be continued to be built on, and which technologies will not properly scale in a multi-machine architecture. As multi-machine systems continue to develop, it is clear that task and fleet management will start to merge into integrated mission management, with customized machine system workflows that optimize the user experience, and integration at the cloud level to share data across systems that can provide value to the end-user. JCA’s technology has been shaped across many agricultural machine systems towards multi-machine systems for both operator-driven and autonomous machines. Our approach in partnering with OEMs with customized machine control applications through the use of technology building blocks has allowed us to adapt to the evolving needs of the market, enabling innovative OEMs to harness emerging technology and create new value for customers in their markets.
Ready to talk?
Need help understanding how you can take advantage of these opportunities for your product? Talk to one of our experts and we can guide you through shifting from task management, fleet management and autonomous mission management into a multi machine system approach.
About JCA Technologies
JCA Technologies was established in 2002 and has engineering and manufacturing facilities in Canada and the US. We partner with agricultural mobile machine OEMs to develop autonomous and highly-automated connected machines. We combine our technology building blocks with engineering and manufacturing capabilities to provide electrical, electronics, and software systems as part of customized mobile machine applications for innovative OEMs.