Skip to content

How can I avoid request parameter names getting encoded when making a form-urlencoded POST request? #1407

New issue
New issue
Open
Open
How can I avoid request parameter names getting encoded when making a form-urlencoded POST request?#1407
Labels
Needs Info
@RacZo

Description

@RacZo
RacZo
opened on Dec 24, 2015

Hello all!

First of all, RetroFit and RetroFit 2.0 are awesome, I've used them in several Android Apps in the past couple of years with great success, well, those Apps had decent backend APIs... but now... I'm working with this horribly designed API (made with ruby on rails) and I can't change the API. This nightmarish API has (among other awful things) POST methods with parameters such as:

  • user[email]
  • location[address]
  • location[city]

And I've created this method to consume one of the methods:

@FormUrlEncoded
@POST("userExists.json")
Call<ExistsResponse> emailExists(@Field("user[email]") String email);

My service is created with a GsonConverterFactory with the following GSON object:

            Gson gson = new GsonBuilder()
                    .excludeFieldsWithModifiers(Modifier.TRANSIENT)
                    .setDateFormat(WebService.API_DATE_FORMAT)
                    .disableHtmlEscaping()
                    .create();

I read that using disableHtmlEscaping() on the gson object would help but it didn't.

The problem I'm having is that the square brackets in the parameter name is getting encoded like this:

... D/OkHttp: user%5Bemail%5D=email%40example.com

(Yes!, I'm using the very neat HttpLoggingInterceptor to log the body of the requests!)

This is driving me crazy, I've tried all the possible ways to make this request and the backend API keeps sending me 404 because it is not understanding the request parameter names.

Is there a way to tell retrofit not to encode the parameter names in the body of a post request?

Your help will be greatly appreciated!

Happy holidays!

Activity

RacZo
changed the title [-]Hw can I avoid request parameter to get encoded when making a form-urlencoded POST request?[/-] [+]Hw can I avoid request parameter names to get encoded when making a form-urlencoded POST request?[/+] on Dec 24, 2015
RacZo
changed the title [-]Hw can I avoid request parameter names to get encoded when making a form-urlencoded POST request?[/-] [+]Hw can I avoid request parameter names getting encoded when making a form-urlencoded POST request?[/+] on Dec 24, 2015
artem-zinnatullin

artem-zinnatullin commented on Dec 24, 2015

@artem-zinnatullin
artem-zinnatullin
on Dec 24, 2015
Contributor

AFAIK: Passed string value is not going through converter (Gson) since it's
a Form parameter. Try @field(encoded = true)

On Thu, Dec 24, 2015, 05:03 Oscar S. notifications@github.com wrote:

Hello all!

First of all, RetroFit and RetroFit 2.0 are awesome, I've used them in
several Android Apps in the past couple of years with great success, well,
those Apps had decent backend APIs... but now... I'm working with this
horribly designed API (made with ruby on rails) and I can't change the API.
This nightmarish API has (among other awful things) POST methods with
parameters such as:

  • user[email]
  • location[address]
  • location[city]

And I've created this method to consume one of the methods:

@FormUrlEncoded
@post("userExists.json")Call emailExists(@field("user[email]") String email);

My service is created with a GsonConverterFactory with the following GSON
object:

        Gson gson = new GsonBuilder()
                .excludeFieldsWithModifiers(Modifier.TRANSIENT)
                .setDateFormat(WebService.API_DATE_FORMAT)
                .disableHtmlEscaping()
                .create();

I read that using disableHtmlEscaping() on the gson object would help but
it didn't.

The problem I'm having is that the square brackets in the parameter name
is getting encoded like this:

... D/OkHttp: user%5Bemail%5D=email%40example.com

(Yes!, I'm using the very neat HttpLoggingInterceptor
https://github.com/square/okhttp/tree/master/okhttp-logging-interceptor
to log the body of the requests!)

This is driving me crazy, I've tried all the possible ways to make this
request and the backend API keeps sending me 404 because it is not
understanding the request parameter names.

Is there a way to tell retrofit not to encode the parameter names in the
body of a post request?

Your help will be greatly appreciated!

Happy holidays!

—
Reply to this email directly or view it on GitHub
#1407.

@artem_zin

RacZo
changed the title [-]Hw can I avoid request parameter names getting encoded when making a form-urlencoded POST request?[/-] [+]How can I avoid request parameter names getting encoded when making a form-urlencoded POST request?[/+] on Dec 24, 2015
RacZo

RacZo commented on Dec 24, 2015

@RacZo
RacZo
on Dec 24, 2015
Author

Digging around, I found this commit 4c38147 made by @JakeWharton that adds methods to tell if the field name or value should be encoded or not in the body.

But, I can't find a way to use them... will keep trying.

UPDATE: Ok... apparently this worked at some point in retrofit one, not retrofit2. :(

JakeWharton

JakeWharton commented on Dec 24, 2015

@JakeWharton
JakeWharton
on Dec 24, 2015
Collaborator

There is a test case that proves encoded=true works:

retrofit/retrofit/src/test/java/retrofit2/RequestBuilderTest.java

Lines 1488 to 1498 in 90729eb

@Test public void formEncodedWithEncodedNameFieldParam() {
class Example {
@FormUrlEncoded //
@POST("/foo") //
Call<ResponseBody> method(@Field(value = "na%20me", encoded = true) String foo) {
return null;
}
}
Request request = buildRequest(Example.class, "ba%20r");
assertBody(request.body(), "na%20me=ba%20r");
}
. What output do you get when you use it?

JakeWharton
added
Needs Info
on Dec 24, 2015
RacZo

RacZo commented on Dec 24, 2015

@RacZo
RacZo
on Dec 24, 2015
Author

Hey thank you @JakeWharton, you rock man!,

Evidently I missed that test case. I tried with:

@Field(value = "user[email]", encoded = false)

and the request didn't went through, then I tried with:

@Field(value = "user[email]", encoded = true)

and that was all I had to do in order to get a response from this API.

On the other hand, there seems to be an issue with the HttpLoggingInterceptor because the output in my log is still:

... D/OkHttp: user%5Bemail%5D=email%40example.com
hatcher521

hatcher521 commented on Jan 8, 2016

@hatcher521
hatcher521
on Jan 8, 2016

i have meet this question,too.forexample:Date type，brackets

JakeWharton

JakeWharton commented on Jun 3, 2016

@JakeWharton
JakeWharton
on Jun 3, 2016 · edited by JakeWharton
Collaborator

Stop spamming our issues. This is your one and only warning.

(Their comment was since deleted)

On Fri, Jun 3, 2016 at 12:41 AM AMIT SHEKHAR notifications@github.com
wrote:

You can use this library . This library supports this
Android Networking
https://github.com/amitshekhariitbhu/AndroidNetworking

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#1407 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAEEEX_rEgq_ZiW4cDdz0WLykbVoc4pgks5qH7CDgaJpZM4G64vQ
.

sriramji

sriramji commented on Nov 10, 2016

@sriramji
sriramji
on Nov 10, 2016

I am facing the same issue after upgrading to Retrofit 2.0

Tried Both :
@field(value = "email", encoded = false) String email
@field(value = "email", encoded = true) String email

Getting same result "sriramji.k**%40**gmail.com", but i want "sriramji.k@gmail.com"

sriramji

sriramji commented on Nov 22, 2016

@sriramji
sriramji
on Nov 22, 2016

@JakeWharton Please help me out of this issue

sriramji

sriramji commented on Nov 29, 2016

@sriramji
sriramji
on Nov 29, 2016

This issue can be fix if you add one more method in Formbody.Builder like

public static final class Builder {
......
public Builder addPlainText(String name, String value) {
      names.add(name);
      values.add(value);
      return this;
    }
.....
}
mzander

mzander commented on Dec 19, 2016

@mzander
mzander
on Dec 19, 2016 · edited by mzander

I ran into the same issue since I upgraded from retrofit 1.9 to retofit 2.

@Field(value = "videos[]") List<Integer> videoIds
Result: videos%5B%5D=4340934

@Field(value = "videos[]", encoded = true) List<Integer> videoIds
Result: videos%5B%5D=4340934

I am not sure how the encoded parameter works but I guess this is only for the values of the field?

@sriramji did you find any solution for this?

sriramji

sriramji commented on Dec 19, 2016

@sriramji
sriramji
on Dec 19, 2016

@mzander For now i created my own FormBody.Builder like i said before

 @POST(LOGIN_URL)
 Call<BaseResponse> loginUser(@Body RequestBody body);

 RequestBody formBody = new FormBody.Builder()
                .add(EMAIL, emailId.getText().toString())
                .add(PASSWORD, password.getText().toString())
                .build();

 Call<BaseResponse> loginCall = RetorfitService.service.loginUser(formBody);

11 remaining items

sriramji

sriramji commented on Sep 15, 2017

@sriramji
sriramji
on Sep 15, 2017

yeah that was my added method. It is not recommended but I have no other choice, so i created my own custom FormBody class

This is the class that we need to look



package okhttp3;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import okhttp3.internal.Util;
import okio.Buffer;
import okio.BufferedSink;

public final class FormBody extends RequestBody {
    private static final MediaType CONTENT_TYPE = MediaType.parse("application/x-www-form-urlencoded");
    private final List<String> encodedNames;
    private final List<String> encodedValues;

    private FormBody(List<String> encodedNames, List<String> encodedValues) {
        this.encodedNames = Util.immutableList(encodedNames);
        this.encodedValues = Util.immutableList(encodedValues);
    }

    public int size() {
        return this.encodedNames.size();
    }

    public String encodedName(int index) {
        return (String)this.encodedNames.get(index);
    }

    public String name(int index) {
        return HttpUrl.percentDecode(this.encodedName(index), true);
    }

    public String encodedValue(int index) {
        return (String)this.encodedValues.get(index);
    }

    public String value(int index) {
        return HttpUrl.percentDecode(this.encodedValue(index), true);
    }

    public MediaType contentType() {
        return CONTENT_TYPE;
    }

    public long contentLength() {
        return this.writeOrCountBytes((BufferedSink)null, true);
    }

    public void writeTo(BufferedSink sink) throws IOException {
        this.writeOrCountBytes(sink, false);
    }

    private long writeOrCountBytes(BufferedSink sink, boolean countBytes) {
        long byteCount = 0L;
        Buffer buffer;
        if(countBytes) {
            buffer = new Buffer();
        } else {
            buffer = sink.buffer();
        }

        int i = 0;

        for(int size = this.encodedNames.size(); i < size; ++i) {
            if(i > 0) {
                buffer.writeByte(38);
            }

            buffer.writeUtf8((String)this.encodedNames.get(i));
            buffer.writeByte(61);
            buffer.writeUtf8((String)this.encodedValues.get(i));
        }

        if(countBytes) {
            byteCount = buffer.size();
            buffer.clear();
        }

        return byteCount;
    }

    public static final class Builder {
        private final List<String> names = new ArrayList();
        private final List<String> values = new ArrayList();

        public Builder() {
        }

        public FormBody.Builder add(String name, String value) {
            this.names.add(HttpUrl.canonicalize(name, " \"':;<=>@[]^`{}|/\\?#&!$(),~", false, false, true, true));
            this.values.add(HttpUrl.canonicalize(value, " \"':;<=>@[]^`{}|/\\?#&!$(),~", false, false, true, true));
            return this;
        }

        public FormBody.Builder addEncoded(String name, String value) {
            this.names.add(HttpUrl.canonicalize(name, " \"':;<=>@[]^`{}|/\\?#&!$(),~", true, false, true, true));
            this.values.add(HttpUrl.canonicalize(value, " \"':;<=>@[]^`{}|/\\?#&!$(),~", true, false, true, true));
            return this;
        }

        public FormBody build() {
            return new FormBody(this.names, this.values);
        }
    }
}

In Builder Inner class add your custom method

public FormBody.Builder addPlainText(String name, String value) {
            this.names.add(name);
            this.values.add(value);
            return this;
}
hubangmao

hubangmao commented on Dec 21, 2017

@hubangmao
hubangmao
on Dec 21, 2017

The problem was finally solved ！
image

add @headers("Content-Type:application/x-www-form-urlencoded; charset=utf-8")

KomoriWu

KomoriWu commented on Feb 2, 2018

@KomoriWu
KomoriWu
on Feb 2, 2018

还是不行

jial-apa

jial-apa commented on Feb 2, 2018

@jial-apa
jial-apa
on Feb 2, 2018

Neither the extra content-type header nor setting the encoded to false works. The fields still got encoded.

michaelwiles

michaelwiles commented on Feb 13, 2018

@michaelwiles
michaelwiles
on Feb 13, 2018

The Field(encoded = true) directive is ONLY there for the scenario that the value you're sending is already encoded. Setting encoded = true does NOT disable encoding it simply ensures the value is not double encoded.

Unfortunately though, as has been mentioned, a lot of people want to turn the encoding off altogether. focussing on the behaviour of the encoded option in the Field is not the place to get this as this is not by design, for turning the encoding off - it is only to stop double encoding.

A feature request should rather be logged to add a flag to turn off encoding altogether via a @field annotation.

pmashelkar

pmashelkar commented on May 8, 2018

@pmashelkar
pmashelkar
on May 8, 2018

I am passing a complex POJO
@post("project/{pro_id}/sender")
Single uploadRenderingJSON(
@path("pro_id") String proId, @Body RenderRequest renderRequest);

Some fields of the POJO are already encoded using StringEscapeUtils.escapeJava(textCaption);

How can I avoid the strings getting encoded again. Since double quotes and backslash are converted to \" and \\ respectively.

Please suggest.

yusufonderd

yusufonderd commented on Jul 15, 2018

@yusufonderd
yusufonderd
on Jul 15, 2018

I'm facing same situation.

BoukhariAyoub

BoukhariAyoub commented on Jul 17, 2018

@BoukhariAyoub
BoukhariAyoub
on Jul 17, 2018

I'm having this problem and nothing of the solutions above is working

Sainathhiwale

Sainathhiwale commented on Jul 27, 2018

@Sainathhiwale
Sainathhiwale
on Jul 27, 2018

[
{
"ofsNo": "180007",
"dispatchDate": "07/04/2018",
"vehicleNo": "ka45p3654",
"transporterName": "trns",
"depotName": "KSBCL",
"ofsId": 1
},
{
"ofsNo": "180004",
"dispatchDate": "07/04/2018",
"vehicleNo": "KA09B6435",
"transporterName": "trns",
"depotName": "KSBCL",
"ofsId": 10006
}
]

Hi every one
i want get ofsno in spinner and base on ofsno populate value into EditText like as
"dispatchDate": "07/04/2018",
"vehicleNo": "KA09B6435",
"transporterName": "trns",
"depotName": "KSBCL", etc
but using retrofit

public void networkCall(){
ofsworld = new ArrayList();
// Create an array to populate the spinner
ofsIdArrayList = new ArrayList();
final ApiInterface apiInterface = ApiClient.getClient().create(ApiInterface.class);
Call<List> call =apiInterface.getOfsID();
call.enqueue(new Callback<List>() {
@OverRide
public void onResponse(Call<List> call, Response<List> response) {
if (response.isSuccessful()){
Log.d("data",String.valueOf(response));
}

           try {
                List<OfsId> ofsIds = response.body();
                for (int i=0;i<ofsIds.size();i++){
                    if (i==0){
                        //String code = ofsIds.get(i).getOfsNo();
                        String leaveType = ofsIds.get(i).getOfsNo();
                        String dipatchDate = ofsIds.get(i).getDispatchDate();
                        String depotName = ofsIds.get(i).getDepotName();
                        String vechicleNo = ofsIds.get(i).getVehicleNo();
                        String trnsName = ofsIds.get(i).getTransporterName();
                        ofsIdArrayList.add(leaveType);
                        et_DispatchDate.setText(dipatchDate);
                        et_DepotName.setText(depotName);
                        et_VechicleNo.setText(vechicleNo);
                        et_TransporterName.setText(trnsName);


                    }


                    arrayAdapter = new ArrayAdapter(MainActivity.this,android.R.layout.simple_spinner_item,ofsIdArrayList);
                    arrayAdapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
                    ofs_Spinner.setAdapter(arrayAdapter);
                }

            }catch (Exception e){
                e.printStackTrace();
            }
        }

        @Override
        public void onFailure(Call<List<OfsId>> call, Throwable t) {

        }
    });
}
394090466

394090466 commented on Feb 26, 2019

@394090466
394090466
on Feb 26, 2019

There is still no solution

sixangle

sixangle commented on Jun 4, 2019

@sixangle
sixangle
on Jun 4, 2019 · edited by sixangle

I have tried most of the solutions above, but no one works. It took me almost half of the day to finally resolve this problem.

Api interface : (Note: do not use the @FormUrlEncoded annotation.)

@POST("/jdai/snapshop")
@Headers("Content-Type:text/plain")
Call<JDAiResultBean> postServer(@QueryMap Map<String, String> queryMap, @Body RequestBody body);

Build your post parameters like as below.

String url = "channel_id=" + CHANNEL_ID  + "&imgBase64=" + mImageBase64 + "&topK=1";
MediaType mediaType = MediaType.parse("text/plain");
RequestBody body = RequestBody.create(mediaType, url);
api.postServer(queryMap, body);

That's all. Hope this could save your time.

FrancisRR

FrancisRR commented on Oct 30, 2019

@FrancisRR
FrancisRR
on Oct 30, 2019

Please Use encoded = true
@query(value = "email", encoded = true)

@http(method = "DELETE", path = "/endurl", hasBody = true)
Observable deleteContact(@query(value = "email", encoded = true) String email);

drod3763

drod3763 commented on Dec 14, 2020

@drod3763
drod3763
on Dec 14, 2020

I am running into this now, and none of the solutions are working. It's encoding an email and password in a form and now it's throwing errors.

power721
mentioned this on Dec 31, 2020
SmartAppsDevelopment

SmartAppsDevelopment commented on Jan 17, 2023

@SmartAppsDevelopment
SmartAppsDevelopment
on Jan 17, 2023 · edited by SmartAppsDevelopment

i have tried all above solution no solution worked for my problem
i am sending encrypted params in POST request witch add special character during encoding and OKHTTP adding special codes as mention above

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Needs Info

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @JakeWharton@RacZo@artem-zinnatullin@michaelwiles@mzander

        Issue actions
          How can I avoid request parameter names getting encoded when making a form-urlencoded POST request? · Issue #1407 · square/retrofit