GA1395 - SkyDemon ‘Flyable Conditions’ feature reliability
Initial Report
Report summary: A GA pilot raised concerns about the reliability of the “Flyable Conditions” feature within SkyDemon. On two recent occasions, the “Flyable Conditions” overlay differed significantly from actual weather observations, both visually and as reported in METARs. On the latter occasion, the SkyDemon support team acknowledged an error in the feed from the forecast provider and advised that a fix had been applied. However, no subsequent explanation was provided as to what safeguards or quality assurance measures are in place to identify such issues proactively or prevent recurrence. The reporter is concerned that, without such measures, incorrect or misleading information may continue to be presented to users without clear caveats.
In particular, the reporter queried whether it would be more appropriate either to:
- Include a more explicit disclaimer regarding the limitations of the “Flyable Conditions” data, particularly in terms of accuracy and validation; or
- Remove the feature altogether if its reliability cannot be assured.
comments
CHIRP Action
CHIRP approached SkyDemon on the reporter’s behalf and asked the following questions:
- How is forecast data for “Flyable Conditions” sourced and processed, and what checks (if any) are applied before it is displayed in the App?
- How are errors identified and addressed, and is there a means of alerting users when data is known or suspected to be inaccurate?
Are there plans to improve the reliability or presentation of this feature, or to consider including stronger caveats about its limitations?
SkyDemon Comment
We have reviewed the notice displayed to pilots when they first use our Flyable Conditions feature (attached the English version) along with relevant pages in the user guide it alludes to (41, 42, 43).
Notice reads:
“This layer is a forecast showing ceiling and visibility over a wide area. It is coloured according to your own preferences for visual conditions. It does not include rain, storms or other significant weather.
For more information on how to use this layer to help you decide where to fly, including safety best practices, consult the user guide.
Hold a finger on the information at the bottom of the map to change your preferences.”
We’re satisfied that we make clear that the feature in question is a forecast and go into some detail on how it works and how to set it up to deliver the personalised graphics it’s designed to produce. We also explain why it may differ from METARs from time to time. Most of our users have gone through PPL training, and for that reason we don’t think it is necessary to explain what a forecast is, or labour the point that a forecast can (and often will) differ from the actual conditions that transpire to some degree.
We did have an issue with the feature in April. It turned out that a mistake by our suppliers resulted in incorrect data being delivered to us. Once we became aware of the problem, we quickly determined the nature of it and reported it to them for a swift fix. Importantly, the system failed safe by design: it erroneously told users that some areas were not flyable, when in fact they were. We do of course have the ability to stop the supply of the data if that should ever become necessary. We had detailed follow-up discussions with our supplier, as you would expect, to establish exactly what went wrong and how, and have satisfied ourselves that they have put measures in place to minimise the chance of it ever occurring again.
CHIRP Comment
Credit to SkyDemon for this comprehensive explanation and for designing the system to ‘fail safe’. This report usefully highlights the potential safety risk of over-reliance on electronic planning and flight aids without adequate cross-checking. SkyDemon has become an indispensable tool for many GA pilots, but like all software it’s only as reliable as its data inputs and the user’s understanding of its limitations. In this case, the anomaly was spotted because the reporter cross-checked multiple sources, something that not every pilot may do.
Instructors increasingly report pilots arriving for club check-outs or lessons having based their planning solely on “what SkyDemon says”. While the software is an excellent aid to situational awareness and can help mitigate risks such as airspace infringements, it is not a substitute for independent weather assessment and sound pre-flight preparation. Features such as “Flyable Conditions” can be helpful visual indicators, but they are still forecasts, subject to the same uncertainties as any meteorological product. On marginal weather days especially, a display that suggests “flyable” could unconsciously bias a pilot’s decision-making towards launching when conditions merit more caution.
Good airmanship means using technology as a tool, not a decision-maker. Pilots should continue to interpret TAFs, METARs and other weather data themselves, comparing outputs from different sources and checking how well automated features align with actual conditions. That human layer of sense-checking and scepticism is what keeps electronic assistance working for us rather than in place of us.
Key Issues relating to this report
Human Factors Considerations
The following Human Factors were central to CHIRP’s discussions on this report:
- Complacency – trusting an automated system without sufficient verification.
- Situational Awareness – maintaining a full picture of weather and operational risk using multiple data sources.
- Decision-making – resisting confirmation bias when an app’s output aligns with a desired outcome.
- Knowledge & Skill – maintaining proficiency in interpreting meteorological data independently.
- Communication – encouraging instructors and peers to discuss and challenge over-reliance on automation.
- Safety Mindset – applying critical thinking and continual cross-checking to validate electronic information.