HttpClient - Unable to write data to the transport connection: The socket has been shut down.

Hello,

I was given an application to debug lately and I encountered an error while sending a json. This request uses HttpClient, the content of the sending is 18Mb and contains 88 photos in base 64 in an HTML structure, in JSON. On the server, the php.ini is configured to accommodate requests of 100Mb max, so I do not think the error comes from the server.
The application makes a post data and throws an exception to the middle of the mailing.

Code:

        if (client.BaseAddress == null)
        {
            client.BaseAddress = new Uri(App.CommonDatas.API_URI);
            client.DefaultRequestHeaders.Accept.Clear();
            client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
            client.DefaultRequestHeaders.Add("Computer-id", CrossDeviceInfo.Current.Id);
        }

        if(!prevent_token && (need_token || app.token != null))
        {
            client.DefaultRequestHeaders.Remove("X-Token");
            client.DefaultRequestHeaders.Add("X-Token", app.token);
        }
        response = await client.PostAsync(path, new StringContent(jsonData))

StackTrace:

+       e   {System.Net.Http.HttpRequestException: An error occurred while sending the request ---> System.IO.IOException: Unable to write data to the transport connection: The socket has been shut down. ---> System.Net.Sockets.SocketException: The socket has been shut down
  at System.Net.Sockets.Socket.EndSend (System.IAsyncResult asyncResult) [0x00012] in <3e9b3e26c4694baab3f689687ad40612>:0 
  at System.Net.Sockets.NetworkStream.EndWrite (System.IAsyncResult asyncResult) [0x00057] in <3e9b3e26c4694baab3f689687ad40612>:0 
   --- End of inner exception stack trace ---
  at System.Net.Sockets.NetworkStream.EndWrite (System.IAsyncResult asyncResult) [0x0009b] in <3e9b3e26c4694baab3f689687ad40612>:0 
  at System.IO.Stream+<>c.<BeginEndWriteAsync>b__53_1 (System.IO.Stream stream, System.IAsyncResult asyncResult) [0x00000] in <fe08c003e91342eb83df1ca48302ddbb>:0 
  at System.Threading.Tasks.TaskFactory`1+FromAsyncTrimPromise`1[TResult,TInstance].Complete (TInstance thisRef, System.Func`3[T1,T2,TResult] endMethod, System.IAsyncResult asyncResult, System.Boolean requiresSynchronization) [0x00000] in <fe08c003e91342eb83df1ca48302ddbb>:0 
--- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.MobileAuthenticatedStream+<InnerWrite>d__67.MoveNext () [0x000d3] in <3e9b3e26c4694baab3f689687ad40612>:0 
--- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.AsyncProtocolRequest+<ProcessOperation>d__24.MoveNext () [0x00196] in <3e9b3e26c4694baab3f689687ad40612>:0 
--- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.AsyncProtocolRequest+<StartOperation>d__23.MoveNext () [0x0008b] in <3e9b3e26c4694baab3f689687ad40612>:0 
--- End of stack trace from previous location where exception was thrown ---
  at Mono.Net.Security.MobileAuthenticatedStream+<StartOperation>d__58.MoveNext () [0x001bf] in <3e9b3e26c4694baab3f689687ad40612>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Net.WebRequestStream+<ProcessWrite>d__33.MoveNext () [0x00270] in <3e9b3e26c4694baab3f689687ad40612>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Net.WebRequestStream+<WriteAsyncInner>d__32.MoveNext () [0x001aa] in <3e9b3e26c4694baab3f689687ad40612>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Net.Http.HttpClientHandler+<SendAsync>d__64.MoveNext () [0x0036e] in <25ebe1083eaf4329b5adfdd5bbb7aa57>:0 
   --- End of inner exception stack trace ---
  at System.Net.Http.HttpClientHandler+<SendAsync>d__64.MoveNext () [0x00489] in <25ebe1083eaf4329b5adfdd5bbb7aa57>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Net.Http.HttpClient+<SendAsyncWorker>d__49.MoveNext () [0x000ca] in <25ebe1083eaf4329b5adfdd5bbb7aa57>:0 
--- End of stack trace from previous location where exception was thrown ---
  at Oracio.WebServices+<PostPutAsync>d__6`1[T].MoveNext () [0x007cb] in C:\oracio\android-old\Oracio\Classes\WebServices.cs:209 }  System.Net.Http.HttpRequestException

I can not provide you with json, it is private but I can validate that besides its size it is valid.
If you need information, for versions of packages for example, I am at your disposal.
I thank you in advance.

Answers

  • LandLuLandLu Member, Xamarin Team Xamurai

    Which line throws this error? It seems you have used Sockets to upload your data, and it lost connection at some time.

  • MaxenceSAUNIERMaxenceSAUNIER USMember ✭✭✭

    This is the function:

    public static async Task<Tuple<bool, string>> PostPutAsync<T>(string path_url, T post_contents, string request_type, string filePath = "", int edl_id = -1, Utilisateur user = null)
    {
        try
        {
            if (path_url == "token/generer-token" || path_url == "logs/send-log")
                InitClient(false);
            else
                InitClient();
    
            string path = path_url;
    
            string json = "";
            if (!(post_contents is string))
                json = JsonConvert.SerializeObject(post_contents);
            else
                json = "" + post_contents;
    
            HttpResponseMessage response = new HttpResponseMessage();
            if (filePath != "")
            {
                Dictionary<string, string> datas = new Dictionary<string, string>();
    
                IFile file = await PCLStorage.FileSystem.Current.GetFileFromPathAsync(filePath);
                string htmlEDL = await file.ReadAllTextAsync(); 
    
                if (!string.IsNullOrWhiteSpace(htmlEDL))
                {
                    if (edl_id > -1)
                    {
                        List<EDLVFC> privloc = EDLVFC.getByEDLId(edl_id);
                        List<string> paraphes = new List<string>();
    
                        if (privloc.Count > 0)
                        {
                            if (user != null)
                            {
                                if (!string.IsNullOrWhiteSpace(user.paraphe))
                                {
                                    datas.Add("html", htmlEDL);
    
                                    string paraphesString = "";
                                    foreach (EDLVFC locat in privloc)
                                    {
                                        List<DelvhGtfvs> locatParaphes = DelvhGtfvs.getByEDLId(edl_id, 0, locat.id);
    
                                        if (locatParaphes.Count > 0)
                                        {
                                            DelvhGtfvs hjsklkik = locatParaphes[0];
    
                                            if (hjsklkik != null)
                                            {
                                                paraphesString += hjsklkik.paraphe + ";";
                                            }
                                        }
                                    }
                                    datas.Add("paraphesprivloc", paraphesString);
    
                                    datas.Add("parapheExpert", user.paraphe);
    
                                    List<Photo> photos = Photo.getByIdEDL(edl_id);
                                    string photosString = "";
                                    foreach (Photo photo in photos)
                                    {
                                        IFile iphoto = await FileSystem.Current.LocalStorage.GetFileAsync(photo.photoPath);
    
                                        Stream photoStream = await iphoto.OpenAsync(FileAccess.ReadAndWrite);
    
                                        byte[] filebytearray = new byte[photoStream.Length];
                                        photoStream.Read(filebytearray, 0, (int)photoStream.Length);
    
                                        photo.base64 = Convert.ToBase64String(filebytearray);
    
                                        string title = "";
                                        if (photo.typeLabel.Trim() == "")
                                            title = photo.elementLabel;
                                        else if (photo.elementLabel.Trim() == "")
                                            title = photo.typeLabel;
                                        else
                                            title = photo.typeLabel + "_" + photo.elementLabel;
                                        title = title.Replace(" : ", "_");
                                        title = title.Replace(" ", "-");
    
                                        photosString += title + ":" + photo.base64 + ";";
                                    }
                                    datas.Add("photos", photosString);
    
                                    string jsonData = JsonConvert.SerializeObject(datas);
                                    if (!string.IsNullOrWhiteSpace(jsonData)) 
                                    {
                                        try
                                        {
                                            using (response = await client.PostAsync(path, new StringContent(jsonData)))
                                            {
                                                //Not going to this part of the code;
                                            }
    
                                        }
                                        catch (Exception e)
                                        {
                                            //come her
                                            throw new Exception("test");
                                        }
                                    }
                                }
                                else
                                {
                                    Log.SaveLog("#private");
                                    await App._mainPage.DisplayAlert("Erreur", "#private", "Ok");
                                }
                            }
                            else
                            {
                                Log.SaveLog("#private");
                                await App._mainPage.DisplayAlert("Erreur", "#private", "Ok");
                            }
                        }
                        else
                        {
                            Log.SaveLog("#private");
                            await App._mainPage.DisplayAlert("Erreur", "#private", "Ok");
                        }
                    }
                }
                else
                {
                    Log.SaveLog("#private");
                    await App._mainPage.DisplayAlert("Erreur", "#private", "Ok");
                }
            }
            else
            {
                switch (request_type)
                {
                    case "post":
                        response = await client.PostAsync(path, new StringContent(json));
                        break;
                    case "put":
                        response = await client.PutAsync(path, new StringContent(json));
                        break;
                    default:
                        break;
                }
            }
    
            return new Tuple<bool, string>(response.IsSuccessStatusCode, await response.Content.ReadAsStringAsync());
        }
        catch (HttpRequestException e)
        {
            Log.SaveLog(e.Message);
            return new Tuple<bool, string>(false, e.Message);
        }
    }
    

    Thanks

  • bswanbswan USMember ✭✭

    I'm having similar issues. My web view is closing and I have an httplistener trying to respond but the socket it closed. It crashes my app. Was there ever a solution?

  • ClintStLaurentClintStLaurent USUniversity ✭✭✭✭✭

    Seeing the same thing 22mar2019 with larger files (16mb in this case). Never happens with small (a few k) files.

    Anyone have a clue?

    cc @bswan @LandLu @MaxenceSAUNIER

  • MaxenceSAUNIERMaxenceSAUNIER USMember ✭✭✭

    I not have found solution. I have limit photo number for the moment.

  • ClintStLaurentClintStLaurent USUniversity ✭✭✭✭✭

    Sadly that's not an option in my case.
    I have a single file of 16meg to transmit to a wifi-direct connected device. And get random socket disconnect errors with no determinable pattern.

    A couple times I've seen a deeply nested "null object reference" exception but didn't capture it or get drilled down to it before Visual Studio lost (timed out) connection to it. Making me wonder if some underlying framework mechanism is being disposed of prematurely.

    cc @bswan @LandLu @MaxenceSAUNIER

  • MaxenceSAUNIERMaxenceSAUNIER USMember ✭✭✭

    I had the idea to divide the files into blocks of 5mo and compile post sending to the webservice. I think this solution should work but I have not had time to look into it. We should try.

  • bswanbswan USMember ✭✭
    edited March 25

    Socket Exception Fix
    I have a work around that prevents the socket exception from crashing the app. Note that I am using the NETStandard HTTP Listener version 1.0.3.5 found here (https://github.com/StefH/NETStandard.HttpListener)

    I removed the Nuget reference for the NETStandard HttpListener and pulled in the actual project.

    In the HttpListener Reponse class, there is a try and finally block that does not catch the socket exception. Here is where the crash was occurring for me (line 74 if you have the source)

        private async Task SendMessage()
        {
            var outputStream = OutputStream as MemoryStream;
            outputStream.Seek(0, SeekOrigin.Begin);
    
            var socketStream = _client.GetOutputStream();
    
            string header = $"{Version} {StatusCode} {ReasonPhrase}" + CharConstants.CRLF +
                            Headers +
                            $"Content-Length: {outputStream.Length}" + CharConstants.CRLF +
                            CharConstants.CRLF;
    
            byte[] headerArray = DefaultEncoding.GetBytes(header);
    
            await socketStream.WriteAsync(headerArray, 0, headerArray.Length);
    
            try
            {
                await outputStream.CopyToAsync(socketStream);
            }
            finally
            {
                await socketStream.FlushAsync();
            }
        }
    

    Adding a catch to the try catch block here will catch the exception (System.IO.IOException: Unable to write data to the transport connection: The socket has been shut down). From there you can do what you wish but I ignore the exception as it is due to the user navigating away from the page before it has finished being served.

        private async Task SendMessage()
        {
            var outputStream = OutputStream as MemoryStream;
            outputStream.Seek(0, SeekOrigin.Begin);
    
            var socketStream = _client.GetOutputStream();
    
            string header = $"{Version} {StatusCode} {ReasonPhrase}" + CharConstants.CRLF +
                            Headers +
                            $"Content-Length: {outputStream.Length}" + CharConstants.CRLF +
                            CharConstants.CRLF;
    
            byte[] headerArray = DefaultEncoding.GetBytes(header);
    
            await socketStream.WriteAsync(headerArray, 0, headerArray.Length);
    
            try
            {
                await outputStream.CopyToAsync(socketStream);
            }
            catch (Exception ex)
            {
    
            }
            finally
            {
                await socketStream.FlushAsync();
            }
        }
    

    I have made the Library author aware of the issue.

    Large Files Potential Fix
    I noticed that some web pages I was hosting have a large MP4 file and did not work on iOS.
    iOS was trying to stream the file. The client can try to stream a file by placing a Range key in the headers.
    See https://stackoverflow.com/questions/25898922/handle-range-requests-with-httplistener for how to handle Range requests.

    I do not know if this will solve your problem but I hope it helps!

  • ClintStLaurentClintStLaurent USUniversity ✭✭✭✭✭

    @MaxenceSAUNIER said:
    I had the idea to divide the files into blocks of 5mo and compile post sending to the webservice. I think this solution should work but I have not had time to look into it. We should try.

    If I were going to a web service/server I controlled I'd do that. But since I'm going to a device with an embedded API I don't have that freedom.

    @bswan said:
    Socket Exception Fix
    I have a work around that prevents the socket exception from crashing the app. Note that I am using the NETStandard HTTP Listener version 1.0.3.5 found here (https://github.com/StefH/NETStandard.HttpListener)

    I've been able to trap the exception and catch from within the shared C# so I haven't had to change component packages. At this point I can handle the exception gracefully, display a popup that the transfer failed, delete the partial file on the receiving device, and re-try. I just hate having random behavior and the need for 0:n retries like that.

    Large Files Potential Fix
    I noticed that some web pages I was hosting have a large MP4 file and did not work on iOS.
    iOS was trying to stream the file. The client can try to stream a file by placing a Range key in the headers.
    See https://stackoverflow.com/questions/25898922/handle-range-requests-with-httplistener for how to handle Range requests.

    I do not know if this will solve your problem but I hope it helps!

    I really appreciate you going the extra mile with that. But since I'm not in control of both sides I have to play by the limitations of the device and the manufacturer's API and the methods it exposes. So i can't do as the poster in that thread did: I can't abandon HTTP in favor of TCP. I have a method exposed by the API for uploading a file from the tablet to the device. It takes parameters for a couple things like directory and file name, and a byte[] that is the stream.content of the request.
    {sarcasm} The device manufacturer actually expects your OS and code to work properly {/sarcasm}
    So when the component handling data transport is buggy - its not their problem. And to a great degree I can see their point.

    Using the device's embedded HTTP server and web interface I can upload the same file countless times, using Chrome or Firefox on the tablet. Presumably its the same calls on the device's backend. Which tells me its a problem in the Xamarin HTTPWebClient.

  • MaxenceSAUNIERMaxenceSAUNIER USMember ✭✭✭

    Using the device's embedded HTTP server and web interface I can upload the same file countless times, using Chrome or Firefox on the tablet. Presumably its the same calls on the device's backend. Which tells me its a problem in the Xamarin HTTPWebClient.

    If you think this is a problem with the Xamarin.Forms web client, you can always try to make them a report if none already exist. Keep me informed of the link to this one so that I can follow up, I'm interested. If I find the time, I will bring the items in my possession to the ticket.

  • ClintStLaurentClintStLaurent USUniversity ✭✭✭✭✭

    Well... we all know how that goes. I've lodged countless reproducible issues through Bugzilla and most get no attention, a couple closed with no reason, a couple closed with "that what we expect it to do regardless what the general coding community thinks"... and then 2 get some attention.

    I've seen no difference in how they are handled in the new GIT system. same people different technique for ignoring bug reports.

    And if I can't make a git repo that they can grab, run and reproduce the bug with everything short of a big flashing neon sign saying "See, here, this should be true not false" - then they close it for not being something they will research.

    One bug did I get great attention on, directly, through our MSDN support system which we pay dearly for. @JGoldberger was amazing and helpful and everything someone could ask for in support and help.

    But every other bug I've ever reported through normal channels just left me feeling disgusted about the entire backend department they have handling reports.

  • MaxenceSAUNIERMaxenceSAUNIER USMember ✭✭✭

    They closed the bugzilla platform for some time. Today, they only go through GitHub for reports.
    I presume that if we are able to provide a fake file with the necessary unit tests it would be a plus. I should be able to look into it in 1 to 2 months minimum.
    Since the last updates they are more reactive for the consideration of the problems but one should not expect a resolution within 2 weeks of the report.

    I would just like to know if you have encountered the problem also on emulator or if it is only present on the physical peripherals?
    I have personally noticed the problem only on physical device.

  • ClintStLaurentClintStLaurent USUniversity ✭✭✭✭✭

    @MaxenceSAUNIER said:
    They closed the bugzilla platform for some time. Today, they only go through GitHub for reports.

    Yes. I know. I'm saying when it was the reporting system so many issues were never dealt with.
    Then when they "transferred" several open bugs that I was watching just never made it to the new GitHub mechantism. They just "slipped through the cracks".

    Since the last updates they are more reactive for the consideration of the problems but one should not expect a resolution within 2 weeks of the report.

    Seeing as there are issues without attention after 2 years - I've learned to not expect fast results.

    I have personally noticed the problem only on physical device.

    That's interesting.

    I know I said I had no issues when transferring though the device's Web interface... but there is an interesting development there. When you select a file you get the android popup "Select file the Android way or file way". It always succeeds "the android way" but fails randomly "the file way". Whatever those two are supposed to be.

    In researching that I think I'm getting close. It seems other apps are having issues with files as well, like when you try to attach a file to an email. It looks like after Android 6 getting file data needs to be done as content instead of as file.
    https://techxmag.com/questions/android-attaching-a-file-to-gmail-cant-attach-empty-file/

    But I'm still researching but this information is looking promising.

  • ClintStLaurentClintStLaurent USUniversity ✭✭✭✭✭
    edited March 25

    Son... of... a....
    So after days of fighting this a casual conversation with a co-worker yielded an interesting response..

    Oh, yah... You can't do file transfers in Android using HTTPS - Android doesn't like that. It gets... squirrely - That's what the 'S' is for: Squirrely

    Huh? Naaaa... The guy is nuts. Right?

    Nope. Doing nothing more than taking the S off the endpoint path so it does HTTP instead of HTTPS (the device supports both)... 20 consecutive uploads of the 16mb file all successful.

    Same code that would randomly crash with no pattern... No changes other than going HTTP and its solid as a rock.

    Freaking Android

    PS: Yes, I was already handling all the credentials, ServicePointManager.ServerCertificateValidationCallback and so on. ANd it would work on occasion with HTTPS, if the moon was right and I sacrificed a chicken nugget after spinning in my chair counterclockwise. Point being - same code would work... then not.. then yes.. then no.

  • MaxenceSAUNIERMaxenceSAUNIER USMember ✭✭✭

    Thank you for your inquiry. I'm trying to test this week.

  • bswanbswan USMember ✭✭
    edited March 26

    Glad you got it fixed! Thanks for posting the solution.

    Something else to keep in mind is that you can change your HttpClient implementation. Under the Android Project -> Properties -> Android Options -> Advanced

    You can select the HttpClient implementation from Default, Managed, or Android. SSL and TLS implementation can also be set here.

Sign In or Register to comment.