HttpWebRequest https problems

AdwenAdwen USMember ✭✭

Hi guys,

Setup:
Windows 7
Visual Studio 2015
OS X Yosemite
Xamarin (latest stable version)

I've been trying to solve this issue. I cannot seem to get any solid information on it. But OK in short I've been trying to make this app for (iOS and android) that I go to this (https) URL and fetch the information and display the information (synchronously). Very straightforward and very easy.

Before I went and put too much effort and work into it I decided to make a console application on my windows 7 machine. I wrote a class there and EVERYTHING was working with no issues.

So I started to make the app and I got to a point where I was going use my class I wrote earlier. I was expecting things to go smooth but no! I was hoping for a clear error but no! It just tells me that the request has timeout. It times out at the following line:

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

This is what I'm seeing when I get the exception:

{System.Net.WebException: The request timed out
at System.Net.HttpWebRequest.EndGetResponse (IAsyncResult asyncResult) [0x00046] in /Users/builder/data/lanes/2077/1d27ac2c/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net/HttpWebRequest.cs:940
at System.Net.HttpWebRequest.GetResponse () [0x0000e] in /Users/builder/data/lanes/2077/1d27ac2c/source/maccore/_build/Library/Frameworks/Xamarin.iOS.framework/Versions/git/src/mono/mcs/class/System/System.Net/HttpWebRequest.cs:958

I did try other URLs just to be sure. It seems to be working fine with HTTP URLs but it has some problems with some HTTPS URLs.

I'm not sure if I'm doing something wrong or the mono/xamarin implementation of HttpWebRequest is not done properly because I was expecting no problems at this point.

I tried asynchronously and I'm still waiting for the callback...

I tried to put a trace on webrequest to see if I could gather more info on what is going wrong but I'm not sure how to do that with xamarin.forms.

Any help or information on the matter is very welcome at this moment.

Best Answer

Answers

  • TorbenKruseTorbenKruse DEMember ✭✭✭

    @Adwen

    I did try other URLs just to be sure. It seems to be working fine with HTTP URLs but it has some problems with some HTTPS URLs.

    Only some or does none of your https calls work? Is a valid ssl certificate installed?

    Also, you could try using HttpClient instead of HttpWebRequest - HttpWebResponse directly. It works fine with https for me. Besides that we probably need some more code to help here.

  • NMackayNMackay GBInsider, University mod

    @Adwen

    Are you running in a simulator?

    If so, you may well have a DNS issue with HTTPS calls. HTTP will work okay. You may have to hit your endpoint by IP address in debug mode and by DNS in production.

    You will also have to override the certificate callback since your hitting the service by IP address:

    iOS

    public override bool FinishedLaunching(UIApplication app, NSDictionary options)
            {
                global::Xamarin.Forms.Forms.Init();
    
                if (UIDevice.CurrentDevice.Model.Contains("Simulator") || UIDevice.CurrentDevice.Name.Contains("Simulator"))
                {
                    ServicePointManager.ServerCertificateValidationCallback +=
                            (sender, cert, chain, sslPolicyErrors) => true;
                }
    

    Android

     public class MainActivity : global::Xamarin.Forms.Platform.Android.FormsApplicationActivity
        {
    
            protected override void OnCreate(Bundle bundle)
            {
                base.OnCreate(bundle);
    
                global::Xamarin.Forms.Forms.Init(this, bundle);
    
                if (Build.Brand.Equals("GENERIC", StringComparison.InvariantCultureIgnoreCase))
                {
                    ServicePointManager.ServerCertificateValidationCallback +=
                        (sender, cert, chain, sslPolicyErrors) => true;
                }
    

    Then in my data service I detect if in a sim and set my endpoint accordingly

     public MyDataService()
            {
                var crypto = ServiceLocator.Current.GetInstance<ICryptoService>();
                var handset = ServiceLocator.Current.GetInstance<IHandsetCommService>();
    
                _endpoint = crypto.Decrypt(handset.IsSimulator ? CDevServiceEndpoint : CServiceEndpoint);
    

    I would try the certificate callback workaround 1st. Check the browser on the emulator can hit the https endpoint (I'm assuming), it may well not.

  • AdwenAdwen USMember ✭✭
    edited October 2015

    It's a Xamarin.IOS app. I didn't get a chance to set it up for android yet.

    @NMackay

    Are you running in a simulator?

    Yes. I was running it on a simulator. But Just to be 100% sure I setup my iPhone 6 to see if there was a different. And there wasn't any single diffference. It behaved the same way as the simulator.
    I tried what you wrote but no luck (on simulator or device).

    @TorbenKruse
    Yes. it is a valid official certificate valid until 3/12/2016. I checked it out plenty and there is nothing wrong with it. I cannot do anything on the server side. forgot to add this: for example if i go to https://www.facebook.com or https://www.google.com it doesn't like time out.

    Thanks for your suggestions guys!

    If you have more please let me know.

    I really hope that there is a solution to it. It would be a shame not to see the final product.

  • Taimoor.JanjuaTaimoor.Janjua DEMember ✭✭

    Hi @Adwen,

    Make sure that the ssl port is registered on the machine. with this,

    • netsh http show sslcert

    if not, than run this command with the appropriate app id and cert thumbprint.

    • netsh http add sslcert ipport=0.0.0.0:[Port] certhash=[ThumbPrint] appid={657e94ac-7558-401f-948a-84a779d4b622}

    make sure that the ssl port rule is defined in the firewall settings if enabled.

    Hope it helps. Let me know, in any case.

    Kind Regards

    Muhammad Taimoor Janjua

  • NMackayNMackay GBInsider, University mod

    @Adwen

    As @TorbenKruse mentioned, it might be worth using HttpClient instead.

  • AdwenAdwen USMember ✭✭

    @NMackay

    As @TorbenKruse mentioned, it might be worth using HttpClient instead.

    Yes I did try to httpclient (if I'm not mistaken based on httpwebrequest class). Same result. So I'm wondering what is the difference in implementation between the mono httpwebrequest and the .net httpwebrequest? Or is this an issue that is outside the implementation?

    @Taimoor.Janjua
    I'm not completely sure how to use your solution? i see commands. Where should I run that? Could you please elaborate a bit more how will this get httpwebrequest running the way it should?

    To reiterate guys: I made a windows console (with visual studio 2015on windows 7 machine) with this code and it works with no problems. it connects to the https server get the data with no issues whatsoever. So Why is there a difference now?

  • Taimoor.JanjuaTaimoor.Janjua DEMember ✭✭

    @Adwen

    You can run these commands from command prompt.

    And as you said, it works from your console app. May be the problem is with your certificate or your https port gets blocked through firewall.

    Hope to hear from you soon!

    Kind Regards,

    Taimoor Janjua

  • NMackayNMackay GBInsider, University mod

    @Adwen

    It certainly performs better than the standard Httpclient.

    I was using ModernHttpClient but had to drop it as I needed the ServerCertificateValidationCallback for the simulators and for that you need the Pro version. As their was an alternative I couldn't justify spending an extra $399 to the IT director.

    http://components.xamarin.com/view/modernhttpclient-pro

  • AdwenAdwen USMember ✭✭

    I made some progress but I'm still not there yet. I'm getting the following error:

    CFNetwork SSLHandshake failed (-9806)

    And

    NSURLSession/NSURLConnection HTTP load failed (kCFStreamErrorDomainSSL, -9806)

    There was an inner exception too:

    an SSL error has occurred and a secure connection to the server cannot be made

    At this point I googled a little and I tried some of the solutions but none of them work yet. So if anyone knows more please let me know.

  • AdwenAdwen USMember ✭✭

    Guys I have some bad news. I was using trail version for xamarin.forms and it has expired! So I cannot do anything anymore because even with my starters edition my source code is bigger than allowed for free.

    So I still want to finish atleast tell you guys this is the exact solution. I was expecting it would let me debug in simulator. I just cut you off period. Really disappointed. Because if things would have gone smoothly (the way it should have) I would have finished before the trail have ended. Now I do not want spend money and on something I'm not sure if xamarin can handle (which was not even difficult to achieve in the first place) and i cannot find a straight forward answer on how to proceed.

    Truly disappointed with xamarin business model.

  • Thanks...Taimoor Janjua Work for me!!!!

  • JohnMurrayJohnMurray GBMember ✭✭

    Hi Guys
    I have the same issue - "Error writing headers" when using SSL site

    I am not au fait with the inner workings of certificates but as a commonsense check I tried the url call entered into a browser (on PC returns correct info,on browser on smartphone ditto, Safari on a Mac at first complained about certificate but is now accepting) So it is a problem with SSL call from webclient. I have tried httpclient and webrequest and either get the same 'error writing headers' exception or a timeout.
    Everything worked fine (the apps both ios and android, have been in release for 2 years)until a month or so ago when the server altered its certificates. I have no control over the server but would be interested to understand why the server seems happy to talk to browsers of whatever platform but not apps - is there some exclusion going on here which the operators could change ? Do some certificactes refuse to talk to anything but a browser?

    FWIW I've tried the
    ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;
    and placed it in OnCreate/finishedlaunch Nada!

    Ive tried moving the code around to every conceivable part of the apps - but that seems to be a red herring

    It's been suggested it is a DNS thing so I tried the IP address but got 'could not find the path' - it may be that https://200.300.400.500 is not allowable - if I put the IP address in a browser i do get redirected to the secure site but my code calls an error
    Has anyone got even a hint of a clue - I seem to have exhausted everything - it does look to me like a mono thing
    here's afacsimile of the code (to protect the innocent)

    public static int OGAuthenticateA(string username, string pwd)
    {
    int rtn = 0;
    if (Convert.ToInt32(Activity1.chkwifi().ElementAt(1)) <= 0)
    {
    return rtn;
    }
    // GZWebClient WC = new GZWebClient (); // a subclass which allowed me to set timeout - not cause of problem
    ///the following line removed to oncreate()
    // ServicePointManager.ServerCertificateValidationCallback += (sender, cert, chain, sslPolicyErrors) => true;

       WebClient WC = new WebClient();
            WC.Headers.Add("username", username);
            WC.Headers.Add("password", pwd);
            WC.Headers.Add("User-Agent", "Android");
            //  string URL = "http://www.carlm.com/";
            string mymode = vineGar.un1;
            string URL = "";
                URL = "https://www.marymaryquitecontrary.com";
                URL="https://187.152.258.68";
    
            Dictionary<string, string> Fields = new Dictionary<string, string>();
            string x = "";
            try
            {
                string JSON = WC.DownloadString(URL + "/authenticate.olg");
    
                Fields = vineGar.ParseJSON(JSON);
    

    //etc }
    catch ()
    {}

    Thank in advace
    JM

  • dqmarkdqmark UAMember ✭✭

    @Taimoor.Janjua said:
    Have you registered your https port with the certificate thumbprint??

    And one more thing try ignoring the certificate check with this:

    ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => { return true; };

    place it in your AppDelegate -> finishedlaunching method for Xamarin Forms iOS or in Mainactivity -> OnCreate method in your Xamarin Forms Android project.

    With this you can verify that you can access your service over https. And the issue lies with your certificate or firewall settings.

    Let me know in anycase.

    Kind Regards

    Taimoor Janjua

    Thank you so much, that finally worked!

  • AdwenAdwen USMember ✭✭

    Hi guys,

    Now that I can use xamarin fully again I just wanted to tell you guys that the solution to my problem was using ModernHttpClient. It works fine now for iOS. But I'm having a problem with andriod (with the session is my assumption). I will start a different thread to deal with that problem.

  • LaherAjmaniLaherAjmani USMember

    @Adwen

    Hi Adwen, can you please share the solution using ModernHttpClient for Xamarin.IOS, i am stuck with same problem on ssl and httpwebrequest

  • AdwenAdwen USMember ✭✭

    @LaherAjmani

    There isn't much documentation or clear tutorials on how to do this but you can find bits and pieces on the internet. I think I put a piece of code on this forum:

    url: https://forums.xamarin.com/discussion/comment/192898/#Comment_192898

  • LaherAjmaniLaherAjmani USMember

    @Adwen

    Thanks adwen for the link,
    i have implemented well for sync and asycn both :)

  • joaobp7joaobp7 BRMember ✭✭

    Thank you so much Taimoor, worked perfect!

  • JeffDJeffD CAMember ✭✭

    I have been seeing the exact same problem as @Adwen and @JohnMurray.

    @Taimoor.Janjua On what machine do you expect to run those "netsh" commands? The https-based web services are outside of my direct control, and those commands make no sense in the context of my app deployed on a mobile device running over cellular data.

    I can succesfully POST to those https-based web services from multiple other clients:

    • C# console app ✓
    • Chrome browser "Simple REST Client" extension ✓
    • SoapUI (written in Java) ✓
    • even a Xamarin Droid project using a hand-written WebRequest) ✓

    To me, the error points to, as @JohnMurray suggested, a bug in Xamarin's Mono-Android implementation of HttpClient.

    I have a PCL project with a Droid project. The web services that I need to consume are WSE 3.0 services. (Yes, they are very old, but they are out of my direct control.) I have generated Service References from the .asmx pages, and those Service References work exactly as expected in the Droid project as long as the connection uses http. (In the generated Service Reference code it uses SoapHttpClientProtocol under the covers.) As soon as I try to connect via https it fails, exactly as @Adwen and @JohnMurray described above. The server has a valid, trusted certificate. And the exact same POST calls from any of those clients I listed above work fine over https.

    The failure is inside SoapHttpClientProtocol.Invoke(..) at:

      Stream requestStream = request.GetRequestStream();  // exception
    

    Overriding the ServerCertificateValidation event to return true does not work because it never reaches my delegate.

    We have a considerable number of web services to consume. I do not relish the thought of writing and maintaining a bunch of hand-written calls via ModernHttpClient or WebRequest. I dearly wish to continue using the generated Service References as there is no good reason for them not to work except that there appears to be a bug inside the Mono implementation of HttpClient.

  • JeffDJeffD CAMember ✭✭

    For the interest of anyone following this, our solution is to subclass SoapHttpClientProtocol, re-implement (via new) the Invoke and InvokeAsync methods to make them use ModernHttpClient, and change the generated Reference.cs file for each asmx service to extend our new subclass.

    public abstract class ModernSoapHttpClientProtocol : System.Web.Services.Protocols.SoapHttpClientProtocol
    {
            protected new object[] Invoke(string methodName, object[] parameters) { }
            protected new void InvokeAsync(string methodName, object[] parameters, SendOrPostCallback sopcb, object userState) { }
    }
    
    public partial class MyWebService : ModernSoapHttpClientProtocol { }
    

    Yes, we have to modify the generated Reference.cs, and we'll have to re-modify it if we ever regenerate the Service Reference, but that will be very rare, if ever again.

    The result, for us, is that we can continue to use the generated Service References for our WSE 3.0 services, and we can connect to those services over https because we've swapped in ModernHttpClient, getting around the apparent HttpClient/https bug in Mono.

  • EmanueleSabettaEmanueleSabetta ITBeta ✭✭✭

    I need the ServerCertificateValidationCallback to develop on the simulator with HTTPS.

    Is there any way to avoid spending $399 for the ModernHttpClient Pro?

  • KyawZinThuKyawZinThu USMember

    @Taimoor.Janjua said:
    Have you registered your https port with the certificate thumbprint??

    And one more thing try ignoring the certificate check with this:

    ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => { return true; };

    place it in your AppDelegate -> finishedlaunching method for Xamarin Forms iOS or in Mainactivity -> OnCreate method in your Xamarin Forms Android project.

    With this you can verify that you can access your service over https. And the issue lies with your certificate or firewall settings.

    Let me know in anycase.

    Kind Regards

    Taimoor Janjua

    @Taimoor.Janjua said:
    Have you registered your https port with the certificate thumbprint??

    And one more thing try ignoring the certificate check with this:

    ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => { return true; };

    place it in your AppDelegate -> finishedlaunching method for Xamarin Forms iOS or in Mainactivity -> OnCreate method in your Xamarin Forms Android project.

    With this you can verify that you can access your service over https. And the issue lies with your certificate or firewall settings.

    Let me know in anycase.

    Kind Regards

    Taimoor Janjua

    @Taimoor.Janjua said:
    Have you registered your https port with the certificate thumbprint??

    And one more thing try ignoring the certificate check with this:

    ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => { return true; };

    place it in your AppDelegate -> finishedlaunching method for Xamarin Forms iOS or in Mainactivity -> OnCreate method in your Xamarin Forms Android project.

    With this you can verify that you can access your service over https. And the issue lies with your certificate or firewall settings.

    Let me know in anycase.

    Kind Regards

    Taimoor Janjua

    @Taimoor.Janjua said:
    Have you registered your https port with the certificate thumbprint??

    And one more thing try ignoring the certificate check with this:

    ServicePointManager.ServerCertificateValidationCallback += (sender, certificate, chain, sslPolicyErrors) => { return true; };

    place it in your AppDelegate -> finishedlaunching method for Xamarin Forms iOS or in Mainactivity -> OnCreate method in your Xamarin Forms Android project.

    With this you can verify that you can access your service over https. And the issue lies with your certificate or firewall settings.

    Let me know in anycase.

    Kind Regards

    Taimoor Janjua

    Thanks Janjua, it's work on both.

Sign In or Register to comment.