Post with empty array in form removes the property

[jwverzijden]

Summary

It appears that there is some form of optimization in the form when posting. This is bad for me, it removes properties from the form.
I'm trying to send an object with a property holding an array which could be empty.
Every time the array is empty an MissingPropertyError is being returned which is incorrect as the empty array actually holds value over not having the property at all.

Simplest Example to Reproduce

const request = require( 'request' )
const bodyParser = require('body-parser')
const app = require('express')()

app.use( bodyParser.json() )
app.use( bodyParser.urlencoded( { extended: true } ) )

app.post('/', function( req, res ){
	console.log( 'received:', req.body )
	res.send('ok')
})
app.listen( 53153 )

const req = {
	url: 'http://localhost:53153',
	method: 'POST',
	headers: {
		'Content-Type' : 'application/x-www-form-urlencoded'
	},
	form: {
		emptyArray: [],
		filledArray: [{something:[]}]
	}
}
console.log( 'sending:', req )
request( req , function( err, httpResponse, body ){
	process.exit(0)
})

Expected Behavior

I expected the send object to be received, as it is valid json

$ node index.js
sending: { url: 'http://localhost:53153',
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  form: { emptyArray: [], filledArray: [ { something: [] } ] }
received: { emptyArray: [], filledArray: [ { something: [] } ] }

Current Behavior

Turns out the received object is nothing but an empty object, looking at wireshark the content-length being sent is 0.

$ node index.js
sending: { url: 'http://localhost:53153',
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  form: { emptyArray: [], filledArray: [ { something: [] } ] }
received: {}

adding null to the array changes behavior:

$ node index.js
sending: { url: 'http://localhost:53153',
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  form: { emptyArray: [ null ], filledArray: [ { something: [ null ] } ] }
received: { emptyArray: [ '' ], filledArray: [ { something: [ '' ] } ] }

however adding undefined in the arrays does not change behavior:

$ node index.js
sending: { url: 'http://localhost:53153',
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  form: { emptyArray: [ undefined ], filledArray: [ { something: [ undefined ] } ] }
received: {}

Possible Solution

Context

Your Environment

software	version
request	2.81.0
node	8.3.0
npm	5.1.0
Operating System	Ubuntu 16.04.2 LTS

[2blane]

So, it appears that the JSON object is getting converted into a string and is being set as the body of the request before being sent. This is the default behavior when you specify the 'form' attribute. The method that this library uses to convert JSON to a string removes empty arrays. What you need to do is instead of passing in the 'form' attribute, pass in the 'body' attribute and JSON.stringify your JSON object to convert it into a string. For example:

NOTE: YOU MIGHT HAVE TO MESS WITH THE HEADERS AND USE application/json as the content type. I haven't experimented to see what works.

{
    url: 'http://localhost:53153',
    method: 'POST',
    headers: { 'Content-Type': 'application/json'},
    body: JSON.stringify({ emptyArray: [], filledArray: [ { something: [] } ] })
}

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[adonisv79]

can't believe that none of the folks working on this project have any input on this. the answer above is invalid as it does not understand the use case mentioned. Why are we providing a sample for "application/json" when clearly the question is about" application/x-www-form-urlencoded". This is an issue happening still but sadly is not given attention.

In my test, the undefined and empty arrays will disappear in the results and anything 'null' will appear in the url-encoded-string.

a json of:
{ results: { arr1: [], arr2: undefined, arr3: null, foo: { arr4: [] } } }
will yield a url with only
"?results%5Barr3%5D="

[spacether]

I am not a maintainer of requests but think that this happens because the values None, [], and {} are undefined per rfc6570 and are omitted from serialization in those cases per:
https://datatracker.ietf.org/doc/html/rfc6570#section-2.3

A variable defined as a list value is considered undefined if the list contains zero members. A variable defined as an associative array of (name, value) pairs is considered undefined if the array contains zero members or if all member names in the array are associated with undefined values.

https://datatracker.ietf.org/doc/html/rfc6570#section-3.2.1
A variable that is undefined ([Section 2.3](https://datatracker.ietf.org/doc/html/rfc6570#section-2.3)) has no value and is ignored by the expansion process. If all of the variables in an expression are undefined, then the expression's expansion is the empty string.