Skip to content

ESP 8266 HTTP Client Connection Failure #5552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
6 tasks done
nespotts opened this issue Dec 25, 2018 · 22 comments
Closed
6 tasks done

ESP 8266 HTTP Client Connection Failure #5552

nespotts opened this issue Dec 25, 2018 · 22 comments
Assignees
Labels
component: web type: troubleshooting waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.

Comments

@nespotts
Copy link

Basic Infos

  • This issue complies with the issue POLICY doc.
  • I have read the documentation at readthedocs and the issue is not addressed there.
  • I have tested that the issue is present in current master branch (aka latest git).
  • I have searched the issue tracker for a similar issue.
  • If there is a stack dump, I have decoded it.
  • I have filled out all fields below.

Platform

  • Hardware: ESP-12 (ESP8266)
  • Core Version: 2.4.2 & 2.5.0-beta
  • Development Env: Arduino IDE and Platformio
  • Operating System: Ubuntu 16.04

Settings in IDE

  • Module: Wemos D1 Mini and NodeMCU 1.0
  • Flash Mode:
  • Flash Size: 4MB/1MB SPIFFS
  • lwip Variant: v2 Lower Memory
  • Reset Method: Nodemcu
  • Flash Frequency:
  • CPU Frequency: Tried both 80Mhz & 160MHz
  • Upload Using: Serial
  • Upload Speed: 115200 for testing and 921600

Problem Description

I am having issues sending HTTP GET requests to a GoPro Camera. I have gotten this to work for 3 different GoPro cameras, it is just the newer Hero 5 that is giving me problems. It is probably something simple, but the client connection works and the HTTP requests work right after the wifi connections have been reset on the camera, but if the ESP8266 is reset after a first WiFi connection, it connects to the WiFi fine, but all HTTP requests return a -1 HTTP Code and the client fails to connect.

I've tried calling client.flush(), client.stop() and just about anything else I can think of or find online. The only way to make the commands work again is to reset the WiFi connections on the camera. If I connect using my computer, I have no problem sending the commands from Google Chrome as shown in the attached picture. I've tried using the regular ESP8266WiFi library and the ESP8266HTTPClient library as shown below. Any help would be appreciated! Thanks

10 5 5 9-gp-gpcontrol-command-shutter p 1 - google chrome_006

Arduino IDE Sketch

#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>

// Use WiFiclient class to create TCP connections
WiFiClient client;
HTTPClient http;

const char* ssid     = "GP25000830";
const char* password = "vault9291";

String wifi_ssid = "GP25000830";
String wifi_password = "vault9291"; 

const char* host = "10.5.5.9";
const int httpPort = 80;


String StartURL = "/gp/gpControl/command/shutter?p=1";


void Trigger() {
  if (!client.connect(host, httpPort)) {
    Serial.println("connection failed");
    return;
  }

  //Command for starting recording
  client.print(String("GET ") + StartURL + " HTTP/1.1\r\n" +
               "Host: " + host + "\r\n" +
               "User-Agent: RCRemote\r\n" +
               "Connection: close\r\n\r\n");
  Serial.println("Triggering");
  delay(20);
}


void Trigger_http() {
  http.begin("http://10.5.5.9/gp/gpControl/command/shutter?p=1");
  int httpCode = http.GET();
  Serial.print("Http Code is: ");
  Serial.println(httpCode);
  http.end();
}


void setup() {
  WiFi.disconnect(true);
  WiFi.disconnect();

  Serial.begin(115200);

  // We start by connecting to a WiFi network

  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());
  client.flush();

}


void loop() {

  Trigger();
  delay(5000);
  Trigger_http();
  delay(5000);
}

Debug Messages

This is just the debug messages generated via the Serial Port (Debug Level = HTTP_CLIENT). The sketch output is intermixed with the debug messages.

SDK:3.0.0-dev(c0f7b44)/Core:2.5.0-beta2=20499902/lwIP:STABLE-2_1_2_RELEASE/glue:1.0-4-gc434c6f/BearSSL:2398cc6
Connecting to GP25000830
mode : sta(38:2b:78:04:3b:81)
add if0
.....scandone
state: 0 -> 2 (b0)
.state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt 
.
connected with GP25000830, channel 6
dhcp client start...
....ip:10.5.5.100,mask:255.255.255.0,gw:10.5.5.9
.
WiFi connected
IP address: 10.5.5.100
connection failed
pm open,type:2 0
[HTTP-Client][begin] url: http://10.5.5.9/gp/gpControl/command/shutter?p=1
[HTTP-Client][begin] host: 10.5.5.9 port: 80 url: /gp/gpControl/command/shutter?p=1
[HTTP-Client] failed connect to 10.5.5.9:80
[HTTP-Client][returnError] error(-1): connection refused
Http Code is: -1
[HTTP-Client][end] tcp is closed
connection failed
[HTTP-Client][begin] url: http://10.5.5.9/gp/gpControl/command/shutter?p=1
[HTTP-Client][begin] host: 10.5.5.9 port: 80 url: /gp/gpControl/command/shutter?p=1
[HTTP-Client] failed connect to 10.5.5.9:80
[HTTP-Client][returnError] error(-1): connection refused
Http Code is: -1
[HTTP-Client][end] tcp is closed

If you need any other details, please let me know. Thanks!

@devyte
Copy link
Collaborator

devyte commented Dec 27, 2018

The debug messages show that the WiFiClient connection fails, which is consistent with the following http client refused message. I'd say the cameranis refusing the connection for some reason.
If it works for others and not the hero 5, then I don't see how it can be a core issue. I suggest seeking help to figure it out further at a community forum like esp8266.com or stackoverflow, given that this is an issue tracker and not meant for general support.
Closing.

@devyte devyte closed this as completed Dec 27, 2018
@nespotts
Copy link
Author

@devyte
I've tried asking just about everywhere including Stackoverflow, ESP8266.com, and EverythingESP.com without luck. If it was the camera refusing the connection, why does it work just fine using an internet browser as shown in the above image?

@devyte
Copy link
Collaborator

devyte commented Dec 28, 2018

I said:

If it works for others and not the hero 5, then I don't see how it can be a core issue.

My suggestion: use wireshark and investigate the difference in communication between a browser and the ESP. Then figure out what is needed to get the ESP to succeed like the browser. It could be something as simple as a missing header, or pairing, or something more complex. Or it could be a bug, in which case you have to show what is wrong in the core.

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 28, 2018

You can try using any tcp server like netcat / nc and address your requests to it. It will print the request (from your browser, or from the esp) so you can make the difference.
firefox http://localhost:1234/gp/gpControl/command/shutter?p=1

$ nc -l 1234
GET /gp/gpControl/command/shutter?p=1 HTTP/1.1
Host: localhost:1234
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:64.0) Gecko/20100101 Firefox/64.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1


Then you can use again nc to duplicate the request (esp's or browser's) and send it to your gopro and see the answer.

@nespotts
Copy link
Author

Thanks for the advise. David Gauchard, I used your method and printed out the Get Requests from my browser and from the ESP, noted the differences and adjusted the GET Request from the ESP. I tried the new request with the GoPro and it still failed :( Then I realized that the issue is not with the GET Request, but with the client.connect() statement. It fails here and never gets to the GET request. Is there anything I can change with the client.connect() to prevent it from failing? Thanks

@nespotts
Copy link
Author

@devyte @d-a-v

I've also tried using Wireshark and can successfully capture all the tcp communications when sending the commands from the browser, but do not know how to capture the requests between the esp8266 and the gopro (when connected to the gopro network, wireshark in promiscuous mode does not capture any of the packets).

I think the key to finding the solution is the fact that:

  1. It is an issue with the TCP Client connection/handshake.
  2. If the esp8266 and gopro are connected and communicating successfully (after the gopro wifi has been reset) and I turn the gopro's wifi off first and either leave the esp8266 on OR reset the esp8266, and then turn the gopro wifi back on, it reconnects and http requests work fine. On the otherhand, if i turn the esp8266 off without turning off the gopro wifi first, client connections always fail thereafter.

Does that provide any useful insight? I've been working at this for a long time. Any help is greatly appreciated! Thanks

@d-a-v
Copy link
Collaborator

d-a-v commented Dec 31, 2018

ESPGoodies on my GH page provides tcpdump (local, or remote for wireshark) from inside the esp8266

You can also go into tests/host and try experimental native build for your computer

@nespotts
Copy link
Author

@d-a-v Sorry about my lack of knowledge, but could you explain how to display the tcpdump using either netcat of wireshark. I added the tcpdump code to the esp8266, but I cannot seem to access it. Thanks ahead of time.

@nespotts
Copy link
Author

Another point of interest that may be helpful is I can ping the esp8266 from my computer, but I am not able to ping the gopro itself (all connected to GoPro's AP).

@nespotts
Copy link
Author

Well, I worked at it some more and I was able to see the tcp dump in the terminal with netcat using the following command:

sudo nc 10.5.5.100 2 -vvn | tcpdump -r -

Here is some of the output:

-4:-53:-17.147231 IP 10.5.5.100.54815 > 10.5.5.9.http: Flags [S], seq 19465, win 2144, options [mss 
536,nop,nop,sackOK], length 0
-4:-53:-17.155438 IP 10.5.5.9.http > 10.5.5.100.54815: Flags [S.], seq 281607446, ack 19466, win 14600, options [mss 1460,nop,nop,sackOK], length 0
-4:-53:-17.156866 IP 10.5.5.100.54815 > 10.5.5.9.http: Flags [.], ack 1, win 2144, length 0
-4:-53:-17.158290 IP 10.5.5.100.54815 > 10.5.5.9.http: Flags [P.], seq 1:344, ack 1, win 2144, length 343: HTTP: GET /gp/gpControl/command/sub_mode?mode=1&[!http]
-4:-53:-17.174038 IP 10.5.5.9.http > 10.5.5.100.54815: Flags [.], ack 344, win 15544, length 0
-4:-53:-17.175569 IP 10.5.5.9.http > 10.5.5.100.54815: Flags [P.], seq 1:53, ack 344, win 15544, length 52: HTTP: HTTP/1.1 200 OK

However, it only worked when the client connection was up and running (right after I reset the gopro). Otherwise, it returned this error in the terminal:

(UNKNOWN) [10.5.5.100] 2 (?) : No route to host
 sent 0, rcvd 0
tcpdump: truncated dump file; tried to read 4 file header bytes, only got 0

Here is the output when communicating with the GoPro from the browser:
selection_008
OR
dump.txt

Can anyone tell if there is a useful difference between these two communications?

@nespotts
Copy link
Author

@devyte @d-a-v

I have a new theory that I've been trying to test. The ESP8266 only works on the 2.4GHz Band. The GoPro Hero 5 is dual band (5 & 2.4) with no specific prioritization as seen here:
https://gopro.com/help/articles/question_answer/What-is-the-Camera-Wi-Fi-Frequency

The later GoPero models (6 & 7) allow you to select the WiFi band, but the 5 does not. Unfortunately, I do not have a Hero 6 or 7 to test with (yet). I've looked at the connection properties from my computer (both when the ESP commands are working and not working) and it always shows 2.4GHz. Is there any way that it would be trying to connect to the ESP at 5GHz and the computer at 2.4?

If this is the issue somehow, it would explain a lot of the symptoms... Does anyone have any insight? Or, any way to determine the WiFi frequency or set a preferred frequency on the ESP? I've tried specifying a channel in the WiFi.begin() command, but I realize that it is a subchannel on the 2.4GHz band.

Thanks

@devyte
Copy link
Collaborator

devyte commented Jan 19, 2019

The ESP is 2.4 only.

@nespotts
Copy link
Author

nespotts commented Feb 7, 2019

Unfortunately, I have not solved the issue yet. Does it work for you on the 2.4Ghz, 5Ghz or neither band?

@d-a-v
Copy link
Collaborator

d-a-v commented Feb 7, 2019

If the esp8266 and gopro are connected and communicating successfully (after the gopro wifi has been reset) and I turn the gopro's wifi off first and either leave the esp8266 on OR reset the esp8266, and then turn the gopro wifi back on, it reconnects and http requests work fine. On the otherhand, if i turn the esp8266 off without turning off the gopro wifi first, client connections always fail thereafter.

@nespotts Have you tried using nc as http server (tcp logguer in fact, instead of your gopro), can you print a request coming from your browser, and another one coming from esp, so we can see the differences ?

@nespotts
Copy link
Author

nespotts commented Feb 7, 2019

I've not tried creating a mock http server (instead of GoPro's) with netcat, but that is a good suggestion. I may be able to try that in the next couple days.

However, from what I have observed in the past, it does not appear to be an issue with the http request, but fails even before that, with the client connection and possibly with the WiFi connection (not able to ping ESP). This is why I am looking into the WiFi band issue with the Hero 5. I am receiving a Hero 6 to test today (can select 2.4Ghz or 5Ghz manually).

@d-a-v
Copy link
Collaborator

d-a-v commented Feb 7, 2019

I was focusing on that message cited above:

If the esp8266 and gopro are connected and communicating successfully...

Maybe a "connection: close" or some other http protocol header is not correctly sent / honored

@d-a-v d-a-v added the waiting for feedback Waiting on additional info. If it's not received, the issue may be closed. label Feb 7, 2019
@nespotts
Copy link
Author

nespotts commented Feb 7, 2019

Gotcha, I will definitely try your suggestion. I have experimented with the http protocol headers a decent amount, but if I post it here (using the nc method suggested) you very well may catch something.

@alinscodes
Copy link

alinscodes commented Feb 7, 2019

With the 5hz it won't detect. so I have to manually select the 2.4hz and then it will detect it and connect. After the restart of the gopro it works fine the first time, then it says the connection is made but the http commands not go through. Exactly as you described.

@embedded4ever
Copy link

Hi @devyte, i have the exact problem. Did you find any solution ?

@myinventions
Copy link

Hello,
I'm analyzing same problem with "[HTTP-Client][returnError] error(-1): connection refused" with 2.7.1 version. It presents only in situations of frequent http.get calls.

In my opinion, problem not comes from the server side, but it comes directly from:
HTTPClient::connected() in file ESP8266HTTPClient/src/ESP8266HTTPClient.cpp
and then
WiFiClient::connected() or WiFiClient::available() from SP8266WiFi/src/WiFiClient.cpp

In my console I found:

[HTTP-Client][begin] url: http://myinventions.pl/someApi.php 
[HTTP-Client][begin] host: myinventions.pl port: 80 url: /someApi.php 
[HTTP-Client][sendRequest] type: 'GET' redirCount: 0
no buf for action frame
no buf for action frame
[HTTP-Client] failed connect to myinventions.pl:80
[HTTP-Client][returnError] error(-1): connection refused
[HTTP] inputsAPI GET... failed, error: connection refused

and the no buf for action frame comes from libnet80211.a (NONOSDK).
#7052

@d-a-v
Copy link
Collaborator

d-a-v commented Jun 23, 2020

@myinventions this is a new problem not much encountered "no buf for action frame" (from nonosdk).
Do you think you can

  • open a new issue
  • provide an MCVE

@myinventions
Copy link

OK, I will collect more info and mcve and open new issue. Strange, that "no buf for action frame" is located inside library libnet80211.a but not visible in source codes (but action frame is used in esp_wifi_internal_tx, so maybe memory related problem...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: web type: troubleshooting waiting for feedback Waiting on additional info. If it's not received, the issue may be closed.
Projects
None yet
Development

No branches or pull requests

6 participants