Skip to main content
  1. Posts/

Post Nested Data Structure to the Server Using Requests

··394 words·2 mins·
Python Requests HTTP Flask
Table of Contents

In this post, I will share how to post complex data and decode it in the server side in Python.

The problem
#

To post a simple Python dict with no nested structure to the server, we may use the below code:

payload = {'name': 'john smith', 'age': 20}

r = requests.post("https://httpbin.org/post", data=payload)

By default, when we use requests.post(url, data=payload) to post payload to the server. We can check the header of the HTTP request via the following command:

print(r.request.header)

The Content-Type of this HTTP request will be application/x-www-form-urlencoded by default. To decode the posted dict, if we use Flask, we can use the following code:

from flask import Flask, request


user_request = request.form.to_dict()

When the data structure is simple, it is fine to use the default options. Once there are nested structure in your Python dictionary, the server can not decode the message properly if the client is using this Content-Type, for example, when we have nested dictionary (a 2-D list):

payload = {'matrix': [[1, 2, 3], [4, 5, 6]], 'msg': 'hello'}

In this case, you will lost the data structure since application/x-www-form-urlencoded can note keep the data structure.

Solution
#

There are two solutions here.

Serialize data as json string
#

First, we can encode the complex dictionary into string using json:

# Note that payload must be json-serializable, or you will meet an error.
payload = json.dumps(payload)

r = requests.post(url, data=payload)

On the server side (suppose that we are using Flask), we can decode the string to get the original dict:

import json

from flask import request

# decodes the string into original dictionary
user_request = json.loads(request.data)

Use application/json as Content-Type
#

Second, we can directly tell the server that we are sending data in json format using requests:

payload = {'matrix': [[1, 2, 3], [4, 5, 6]], 'msg': 'hello'}

r = requests.post(url, json=payload)

In this case, the request header will be something like the following:

{'User-Agent': 'python-requests/2.22.0', 'Accept-Encoding': 'gzip, deflate',
'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '2705',
'Content-Type': 'application/json'}

The Content-Type field will be application/json, in which we tell the server that we are sending JSON data.

On the server side (suppose you are using flask), we can retrieve that data using the following script:

from flask import request


# directly decodes the request body as JSON.
user_request = request.get_json()

Ref
#

Related

Build Web API with Flask --- Work with JSON-like Dict
··435 words·3 mins
Python Flask HTTP Requests
Build Web API with Flask -- Post and Receive Image
··657 words·4 mins
Python Flask Requests HTTP
Build A Web API Service Using Flask -- The Basics
··1014 words·5 mins
Python Flask HTTP Requests