Current and Recent Releases: UITest, Test Recorder and Calabash

Glenn.WilsonGlenn.Wilson USXamarin Team Xamurai
edited June 6 in Xamarin Test Cloud

Current and Recent Xamarin Test Cloud Framework Releases

[UITest iOS] Xamarin.TestCloud.Agent 0.20.6 June 5, 2017
https://www.nuget.org/packages/Xamarin.TestCloud.Agent/0.20.6

  • Fixes some ScrollTo(marked) not working iOS 10.

[UITest] Xamarin.UITest 2.0.9 April 27, 2017
https://www.nuget.org/packages/Xamarin.UITest/2.0.9

  • [Android] Update Android server to version 0.9.1 (addresses Context.MODE_WORLD_READABLE removed in Android M).

[UITest] Xamarin.UITest 2.0.8 April 10, 2017
https://www.nuget.org/packages/Xamarin.UITest/2.0.8

  • [iOS] Added support for Xcode 8.3 and iOS 10.3.
  • [iOS] Auto-dismiss the new VPN springboard alert.
  • [iOS] Auto-dismiss springboard alerts for devices configured with Korean language settings.
  • Fix issue where test-cloud.exe could fail with a Test chunking failed: error when submitting tests on Windows.

[Calabash iOS] run_loop 2.4.0 April 7, 2017
https://rubygems.org/gems/run_loop/versions/2.4.0

[Calabash iOS] run_loop 2.3.1 March 7, 2017
https://rubygems.org/gems/run_loop/versions/2.3.1

[Test Recorder] Test Recorder for Visual Studio 0.4.5 March 7, 2017
https://marketplace.visualstudio.com/items?itemName=XamarinInc.XamarinTestRecorder2013
https://marketplace.visualstudio.com/items?itemName=XamarinInc.XamarinTestRecorder2015

  • VS 2017 support (in VS 2015 version).
  • Fix issues with screenshot events not showing up.

[Calabash] xamarin-test-cloud 2.1.2 March 6, 2017
https://rubygems.org/gems/xamarin-test-cloud/versions/2.1.2

  • Fixes profile is not being written to upload manifest during prepare command.

[UITest] Xamarin.UITest 2.0.7 February 10, 2017
https://www.nuget.org/packages/Xamarin.UITest/2.0.7

  • [iOS] test-cloud.exe accepts --language parameter for use with the submit command.

[Calabash] xamarin-test-cloud 2.1.1 February 9, 2017
https://rubygems.org/gems/xamarin-test-cloud/versions/2.1.1

  • Added language parameter.

[UITest] Xamarin.UITest 2.0.6 February 7, 2017
https://www.nuget.org/packages/Xamarin.UITest/2.0.6

  • [iOS] Check for supported version of Xamarin Test Cloud Agent or Calabsh iOS Server.
  • [iOS] iOSApp.ScrollTo() looks for the marked element within each scrollable element on the screen instead of just the fist one. Only one element is scrolled if multiple elements contain the marked element.
  • [iOS] Touch duration in iOS10 increased to 0.2s to match XCUITest.
  • [iOS] Improved reliability of text entry on i386/armv7 devices.
  • [iOS] Additional alerts are automatically dismissed on devices set to nl_BE and sv locales.

[Calabash iOS] run_loop 2.3.0 February 3, 2017
https://rubygems.org/gems/run_loop/versions/2.3.0

  • This release contains a significant change - Simulators will not be automatically relaunched whenever Calabash::Cucumber::Launcher#relaunch is called. Instead, run-loop will try to keep the simulator running as long as it appears to be healthy.
  • Fix text entry for i386/armv7 devices.
  • Various stability improvements.
    More details are at: https://github.com/calabash/run_loop/blob/develop/CHANGELOG.md

[Calabash iOS] calabash-cucumber 0.20.4 January 2, 2017
https://rubygems.org/gems/calabash-cucumber/versions/0.20.4
This release, combined with run-loop 2.2.4, fixes several critical bugs related to:

  • Code signing the DeviceAgent-Runner.app for physical devices
  • Text entry and keyboard interactions
  • Dismissing SpringBoard alerts
  • Defining a CODE_SIGN_IDENTITY is no longer necessary, but is supported if for some reason you require a specific identity for signing.
    More details are at: https://github.com/calabash/calabash-ios/blob/develop/CHANGELOG.md

[UITest] Xamarin.UITest 2.0.5 December 19, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.5

  • Fixed issue where REPL failed with Mono.CSharp assembly loading error on Windows.

[UITest] Xamarin.UITest 2.0.4 December 16, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.4

  • Added overload of AppQuery.Id() which takes an int id parameter. Allows properties of an Android App project's Resource.Id to be used in Id() queries.
  • Fixed issue which caused all test methods to run in xtc when the user specified a specific test using the '--fixture' parameter.
  • Fixed issue which caused the REPL to crash when resizing from a larger window to a smaller window.
  • [iOS] Fixed issue where iOSApp.Device.IsSimulator could be false even though the test was running on a simulator.

[Calabash iOS] run_loop 2.2.4 December 15, 2016
https://rubygems.org/gems/run_loop/versions/2.2.4

[Test Recorder] Test Recorder for Visual Studio 0.4.4 December 12, 2016
https://marketplace.visualstudio.com/items?itemName=XamarinInc.XamarinTestRecorder2013
https://marketplace.visualstudio.com/items?itemName=XamarinInc.XamarinTestRecorder2015

  • Improved error messages when connection fails.
  • Improved compatibility with Xamarin for Visual Studio.

[Calabash Android] calabash-android 0.9.0 December 12, 2016
https://rubygems.org/gems/calabash-android/versions/0.9.0

  • Change Clearing now uses ClearAppData2, which is more identical to reinstalling an application, and only removes accounts belonging to the app under test.
  • Add Method ensure_app_installed which ensures the application and test-server are installed. It does not reinstall if the apps are installed. Use with clear_app_data for drastic speed improvements.
    Example:
     ensure_app_installed
     clear_app_data
  • Fix Unable to parse ADB output if ADB was not already running (first ADB command)
  • Change Use SHA1 output of JDK tool for fingerprinting, as some JDK keytools seem not to output MD5 fingerprints.
  • Change Default skeleton now uses ensure_app_installed/clear_app_data instead of reinstalling.

[UITest] Xamarin.UITest 2.0.3 December 6, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.3

  • [iOS] iOSAppConfigurator.StartApp() will reuse the running simulator if possible.
  • [iOS] Fixed some issues which could cause tests to fail when entering text on iOS 10. Work is continuing to further improve reliability. Issues remain especially on i386/armv7 devices.
  • [iOS] Fixed issue where PinchToZoomIn() and PinchToZoomOut() zoomed much futher with iOS 10 than in previous versions.
  • [Android] Fixed errors when using REPL if an ANDROID_HOME environment variable in the terminal points at an Android SDK incompatible with the Android SDK used by UITest.
  • Modified the REPL to work with Mono 4.8 / Xamarin Studio Alpha / Visual Studio for Mac.

[Calabash iOS] run_loop 2.2.3 December 6, 2016
https://rubygems.org/gems/run_loop/versions/2.2.3.

  • Improvements for entering text. Issues remain especially on i386/armv7 devices.
  • Update DeviceAgent, iOSDeviceManager.
  • CODE_SIGN_IDENTITY is not longer required.
  • Increase DeviceAgent timeouts.
  • DA:Client: launch AUT with args and environment.
  • run-loop simctl tail shows all simulator logs.
  • Support Xcode 8.2.
  • Turns enter text input parameter to string for numeric keyboards.
  • Gem: patch awesome_print for BasicObject.
    More details are at: https://github.com/calabash/run_loop/blob/develop/CHANGELOG.md

[UITest] Xamarin.UITest 2.0.2 November 23, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.2

  • Fixed issue resulting in connection errors when using Mono 4.8 / Xamarin Studio 6.2 / Visual Studio for Mac.

[Test Recorder] Xamarin Test Recorder 1.0.2 November 21, 2016
https://developer.xamarin.com/guides/testcloud/testrecorder/

  • Updated license.

[Test Recorder] Xamarin Test Recorder 1.0.0 November 17, 2016
https://developer.xamarin.com/guides/testcloud/testrecorder/

  • Removes Test Cloud API key dependency.
  • Improved Telemetry and logging.
  • Improved Error messages when connection fails.
  • Fixed Error loading some saved sessions.
  • Fixed Incorrect ScrollTo code generated for some cases.

[UITest] Xamarin.UITest 2.0.1 November 14, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.1

  • Added IApp.SwipeRightToLeft() and IApp.SwipeLeftToRight() for use in web views.
  • [iOS] withInertia can be set to false for Scroll* methods when using iOS10.
  • [iOS] Fix issue where tests fail when using .AppBundle() with paths containing a space.
  • [Android] Workaround for apps where manifest as reported by aapt dump xmltree is invalid.
  • [Android] Android Test Server updated to 0.8.4.
  • [Android] Fixed occasional failure detecting devices after connecting a new device.
  • test-cloud.exe --fixture parameter now also accepts a class name without a namespace.

[Calabash Android] calabash-android 0.8.4 November 7, 2016
https://rubygems.org/gems/calabash-android/versions/0.8.4

  • Fix Calabash-server not connecting to applications that were blocking their handler (this fixes a regression from 0.7.3).
  • Fix Entering/clearing text not working on the latest Android system web view.
  • Add Silence rubyzip invalid date warnings (lastest Android gradle build uses invalid dates).
  • Fix Web queries for SVG elements misbehaving.
  • Fix Web queries for SVG elements misbehaving

[Test Recorder] Xamarin Test Recorder 0.14.0 October 31, 2016
https://developer.xamarin.com/guides/testcloud/testrecorder/

  • Fixed Device connection and discovery issues.
  • Improved iOS view detection.
  • Improved Connection error messages.

[Test Recorder] Xamarin Test Recorder 0.13.0 October 17, 2016
https://developer.xamarin.com/guides/testcloud/testrecorder/

  • iOS 10 support.
  • Improved iOS connection reliability.
  • Fixed detection of taps on UISwitch elements.

[Calabash iOS] calabash-cucumber 0.20.3 October 14, 2016
https://rubygems.org/gems/calabash-cucumber/versions/0.20.3
This release, combined with DeviceAgent 1.0.2, and run-loop 2.2.2, fixes several critical bugs related to:

  • Code signing the DeviceAgent-Runner.app for physical devices
  • Text entry and keyboard interactions

[Calabash iOS] run_loop 2.2.2 October 13, 2016
https://rubygems.org/gems/run_loop/versions/2.2.2

  • Update DeviceAgent, iOSDeviceManager.
  • Fixes for pan gesture.
  • Code signing improvements.
  • Increase timeouts for enter text.
  • Enter text without keyboard check.

[UITest] Xamarin.UITest 2.0.0 October 11, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.0

  • Works with Xcode 7 or Xcode 8.
  • Support for iOS 9 and iOS 10 with Xcode 8.
  • If you need to test iOS 8, you must have Xcode 7 installed. macOS Sierra does not support Xcode 7, so keep that in mind when making your macOS upgrade plans.

  • Breaking Changes

  • [iOS] Running iOS tests is only supported on OS-X El Capitan and newer.
  • Added GestureCompletionTimeout property to IWaitTimes. This can be used to allow slower gestures (e.g. drag) to complete when using a short GestureWaitTimeout value. When adding GestureCompletionTimeout values to custom implementations of IWaitTimes, using the same value as GestureWaitTimeout is likely to be appropriate.
  • InvokeUIA: Since Xcode 8 does not support UI Automation, UITest tests that use app.InvokeUIA methods will fail when using Xcode 8. Since Apple has dropped support for UI Automation in Xcode 8, we do not anticipate InvokeUIA working in the future.

  • Improvements

  • [iOS] Added support for iOS 10, Xcode 8 and OS-X Sierra.
  • [iOS] ClearText() now works for elements within web views.
  • Fix for occasional SocketException failures in Test Cloud.
  • [iOS] Providing global wait times using .WaitTimes() which is part of the ConfigureApp class are now honored in Test Cloud.

  • Known Issues

  • [iOS] EnterText() can occasionally be slow or throw errors.

  • Changes since the previous beta 2.0.0-beta06

  • [iOS] Fixed SetUp : System.NullReferenceException : ... at Xamarin.UITest.iOS.iOSAppLauncher.GetCalabashDevice
  • [iOS] Logging improvements
  • [iOS] Performance improvements

[UITest] Xamarin.UITest 2.0.0-beta06 October 5, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.0-beta06

  • [iOS] CodesignIdentity is no longer used and will be removed soon. An appropriate certificate still must be installed and will now be automatically located.
  • [iOS] Fixed build conflict with TinyIoC.

[UITest] Xamarin.UITest 2.0.0-beta05 September 30, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.0-beta05

  • [iOS] Support for ScrollTo, ScrollDownTo and ScrollUpTo. Requires Xamarin.TestCloud.Agent 0.20.2 or newer.

[Calabash iOS] run_loop 2.2.1 September 30, 2016
https://rubygems.org/gems/run_loop/versions/2.2.1

  • Update DeviceAgent, iOSDeviceManager, FBSimulatorControl
  • iOSDeviceManager crashes during DeviceAgent install (resigning)
  • Entering text causes app crashes

[UITest iOS] Xamarin.TestCloud.Agent 0.20.3 September 30, 2016

  • Update form factor table with latest iPhone 7 device model info.

[UITest iOS] Xamarin.TestCloud.Agent 0.20.2 September 29, 2016

  • Scrolling functionality to support iOS ScrollTo, ScrollDownTo and ScrollUpTo in an upcoming Xamarin.UITest 2.0 release.

[UITest] Xamarin.UITest 2.0.0-beta04 September 28, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.0-beta04

  • [iOS] Fix: *** Assertion failure in -[FBControlCoreConfiguration_Device_iPhone7 productTypes],...
  • [iOS] Fix: SetLocation fails with ConnectFailure.iOS 9.3.5. In test cloud.
  • [iOS] Fix: iOS user defined wait times not honored in XTC.
  • Fix: await Task.Delay() causes any following UITest calls to hang

[UITest] Xamarin.UITest 2.0.0-beta03 September 21, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.0-beta03

  • [iOS] Fix: Unable to contact DeviceAgent Error
  • [Android] Fixed: Test Fails on the SetUp with different System.Exception errors, usually involving adb related error.

[UITest iOS] Xamarin.TestCloud.Agent 0.20.1 September 19, 2016

[Calabash Android] calabash-android 0.8.2 September 15, 2016
https://rubygems.org/gems/calabash-android/versions/0.8.2

  • Fix Some apps take longer than 15 seconds to start-up. The startup timeout is increased to 60 seconds.

[UITest] Xamarin.UITest 2.0.0-beta02 September 14, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.0-beta02

  • Possible fix for SetUp : System.Exception : Failed to execute: ... exit code: 259 error.

[UITest] Xamarin.UITest 2.0.0-beta01 September 13, 2016
https://www.nuget.org/packages/Xamarin.UITest/2.0.0-beta01

  • Initial support for iOS 9 and iOS 10 with Xcode 8.
  • If you need to test iOS 8, you must have Xcode 7 installed. macOS Sierra does not support Xcode 7, so keep that in mind when making your macOS upgrade plans.
  • Works with Xcode 7 or Xcode 8.
    Breaking Changes
  • Added GestureCompletionTimeout property to IWaitTimes. This can be used to allow slower gestures (e.g. drag) to complete when using a short GestureWaitTimeout value. When adding GestureCompletionTimeout values to custom implementations of IWaitTimes, using the same value as GestureWaitTimeout is likely to be appropriate.
  • [iOS] Since Xcode 8 does not support UI Automation, UITest tests that use app.InvokeUIA methods will fail when using Xcode 8.
    Improvements
  • [iOS] ClearText() now works for elements within web views.
    Known Issues
  • [iOS] ScrollTo() is not yet supported when using Xcode 8 (iOS 10 or iOS 9).

[Calabash iOS] calabash-cucumber 0.20.0 September 13, 2016
https://rubygems.org/gems/calabash-cucumber/versions/0.20.0

  • This release provides support for iOS 9 and iOS 10 with Xcode 8.
  • If you need to test iOS 8, you must have Xcode 7 installed. macOS Sierra does not support Xcode 7, so keep that in mind when making your macOS upgrade plans.

[Calabash iOS] run_loop 2.2.0 September 12, 2016
https://rubygems.org/gems/run_loop/versions/2.2.0

  • Update DeviceAgent, iOSDeviceManager, FBSimulatorControl
  • Fix retry with dylib injection and DeviceAgent #534

[UITest iOS] Xamarin.TestCloud.Agent 0.20.0 September 12, 2016
https://www.nuget.org/packages/Xamarin.TestCloud.Agent/0.20.0

  • Update for Xcode 8, iOS 10, and iPhone 7/7+.
  • If you need to test iOS 8, you must have Xcode 7 installed. macOS Sierra does not support Xcode 7, so keep that in mind when making your macOS upgrade plans.

[Calabash iOS] run_loop 2.1.11 September 12, 2016
https://rubygems.org/gems/run_loop/versions/2.1.9

  • Update DeviceAgent, iOSDeviceManager, FBSimulatorControl
  • Respond to DeviceAgent SpringBoard alert route changes #532
  • Support LPServer dylib injection on Simulators when running with DeviceAgent #531

[Calabash iOS] run_loop 2.1.10 September 9, 2016
https://rubygems.org/gems/run_loop/versions/2.1.10

  • Update DeviceAgent, iOSDeviceManager, FBSimulatorControl
  • Allow the user to increase the default DeviceAgent install time #529
  • DA:Client: prototype API for SpringBoard alerts #528
  • CoreSim: don't wait after simctl install #527
  • Improve DeviceAgent::Client HTTP interactions #526
  • DeviceAgent: Cleanup life cycle methods #525
  • DA:Client: increase default timeouts for Xamarin Test Cloud #524
  • DA:IDM: Disable is installed? check for physical devices
  • DA:IDM: only restart the simulator if necessary
  • CLI: simctl tail displays last 5000 lines
  • Expand DeviceAgent query and gestures #522

[Calabash iOS] run_loop 2.1.9 September 3, 2016
https://rubygems.org/gems/run_loop/versions/2.1.9

  • DeviceAgent: replace :gesture_performer with :automator #520

[Test Recorder] Xamarin Test Recorder 0.12.0 August 29, 2016
https://developer.xamarin.com/guides/testcloud/testrecorder/

  • Better ABI match check to avoid false mismatch
  • Fixed Crash detecting some Android devices
  • Fixed some iOS devices were not being detected
  • Updated to Xamarin.UITest 1.3.15
  • Fixed issues with PressEnter() and PressMenu() on Android
  • Fixed detecting physical iOS devices for some users
  • Better error message when adb port forward fails

Posts

  • JKayJKay USMember ✭✭✭

    Are there any plans to add UWP to Xamarin.UITest (being a Microsoft company and all).

    What does the UITest / Testcloud roadmap look like?

  • Kent.GreenKent.Green USXamarin Team Xamurai

    UWP Support was announced at //Build: https://channel9.msdn.com/Events/Build/2017/KEY02#time=1h08m18s

    The Mobile Center preview already has some UWP features, but Test Cloud is still currently omitted. I'm not aware of any announcement of an explicit ETA when that will be ready.

Sign In or Register to comment.