A dev release of Xamarin.UITest with the
App.Repl() fix is now available: https://www.nuget.org/packages/Xamarin.UITest/18.104.22.1684-dev
As a dev build it hasn't fully passed QA, so I'd mainly recommend trying it if you had trouble with the workaround (either implementing it or hitting other issues); or perhaps if you're not in a rush and are just in the mood to try it. If the workaround is currently acceptable to you then I don't think trying to fix this issue with the dev build is likely to be terribly necessary.
It might be more likely to encounter issues than the full release, but please feel free to reach out if you have any observations about using it.
(Edit: I used my admin ability to mark this as an answer as well, because a user found this thread but missed that a fix was available with this version, so I wanted to make it more visible.)
I took a quick look and confirmed a couple of the latest version links are missing or broken; I've passed your feedback on to our team members, my apologies for the inconvenience.
@anonymousisme Unfortunately I don't currently have an ETA on that release.
Incidentally, Calabash Cucumber & Xamarin Test Cloud Agent versions have parity, they're actually the same but packaged differently; Calabash Cucumber is mainly for iOS Native Apps, while Xamarin Test Cloud Agent is for iOS apps written in C#.
I have several updates regarding this issue:
It turns out that
Xamarin.UITest is at fault for the issue, but for an unexpected reason. Basically the Test Cloud Agent had been using a depreciated call and switched to the updated one, but UITest did not switch to the updated version. Because it was the Test Cloud Agent that changed, downgrading it works; but it wasn't technically the bugged tool.
A fix has been made for the issue in Xamarin.UITest and is expected to be released in the next version of the package. (This fix will not trigger an update in Test Cloud Agent or Calabash-Cucumber, since they aren't the tools applying the change.)
The fix has one way compatibility. Test Cloud Agent/Calabash-Cucumber 0.21.4+ will not be compatible to use the
Repl() tool with versions of Xamarin.UITest 2.2.2 or below. But new versions of Xamarin.UITest should still be able to use the
Repl() tool even with
Test Cloud Agent/Calabash-Cucumber 0.21.3-, for at least a little while. So while when the fix comes out ideally get up to date on both of those, Xamarin.UITest is more essential to update.
Unfortunately I still don't have a precise ETA on the release, just that the fix should be in the next stable release of Xamarin.UITest.
Our team worked with a few customers who reached out to Test Cloud support & hit this issue specifically when using the
We were able to reproduce the issue and determined that it is correlated with the Xamarin.TestCloud.Agent 0.21.4 package and have filed a bug report based on that. Downgrading to Xamarin.TestCloud.Agent 0.21.3 is an effective workaround for this problem in our tests; and moreover the Xamarin.UITest package version did not seem to matter with regard to reproducing the issue so long as it was a recent version.
(Note: Xamarin.UITest 2.2.1 or Xamarin.UITest 2.2.2...Being a recent version matters because with all the iOS 11 & Xcode 9 changes that are still a little fresh, there are known issues that might cause other, more serious problems if UITest is significantly out of date; examples are in the release notes for the last few versions of UITest on NuGet.)
According to our tests, the confirmed bug only relates to the issue with the
Repl() tool though; not other types of UITest or query failures. Some types of UITests may still work normally.
While there may be additional issues with Xamarin.TestCloud.Agent 0.21.4 (as indicated by @PeterSimons comment, which is not the same as this issue); we are still investigating those other issues because even if the same downgrade works; we don't have any indication that the fix for the
Repl() issue will fix other symptoms as well.
Yes, if either of these applies to your situation:
Repl()behavior is still not fixed, even after downgrading to TestCloud.Agent 0.21.3 and rebuilding (while keeping the other packages up to date).
Repl()when using the latest Test Cloud packages, including TestCloud.Agent 0.21.4, even if downgrading helps. This is because that's not the same problem, and we may need to investigate it more in-depth to fix it. (Like @PeterSimons correctly did.)
Even though you might only hit the issue locally, our Test Cloud support team investigates potential bugs with local Xamarin.UITest because there is a lot of crossover between supporting the testing platform & the testing framework. You can reach out to support by logging into testcloud.xamarin.com & clicking on "Support" in the menubar: