Summary

Last week, a small number of our customers were affected by a change we made that caused notifications sent to particular device groups to fail to reach their intended on-premises recipients. Although we resolved the problem quickly, sending notifications consistently and accurately is the core of our product, and we realize the impact this had on some of you. We’re very sorry, and we want you to know what went wrong and what we’re doing to prevent it from happening again.

What went wrong?

Device groups allow you to group together on-premises recipients in several ways: by selecting them individually; selecting multiple other, existing groups to combine; or constructing matching rules. Individual devices that would otherwise be included can also be specifically excluded. The logic we use in determining if devices should match is:

(device is in individual additions) OR (device is in any included other existing group AND device matches rules) AND NOT (device is in individual exclusions)

In our release two weeks ago, we made a change to individually added IP speakers and InformaCast Desktop Notifiers that made them behave as if they were rules, rather than individual additions. In doing so, we neglected to consider specific case where a device group had both other existing groups included and specific speakers or InformaCast Desktop Notifiers added. Unfortunately, the result was that neither devices in other existing groups nor the individual speaker nor desktop notifier additions were included.

We have extensive tests meant to catch problems like this, but in the case of this particular problem, while we had separate tests for individual additions, existing device groups, and matching rules, we didn’t have one for this specific combination.

What are we doing about it?

In addition to the fix we released, we’re adding additional tests for device groups, both for this specific case and for other permutations of our logic. These will ensure that the next time we need to tweak this code for whatever reason, we won’t be able to re-introduce this problem or a similar one.

We also realize that the logic of how devices are included is complex and can be difficult to understand. We’re working on changes to the Device Group Details page of our Administration Console to make the interaction between these inclusion methods clearer, both for you and us.