Activity - Import users from a file


Introduction and workflow tasks

This activity shows you how to use the /environments/{environmentId}/importTasks service to bulk import users from a comma-separated value (csv) file. The bulk import workflow for importing users from a file is at minimum a two-step process. First, you need to create a new importTask resource for the import operation. Then, using the Id value of the import task resource you just created, you can submit a POST request to specify the csv file for processing.

During bulk import, the entries in the csv file are treated as command events in which each row of the file is processed as a separate task. The state attribute of each separate command event is catalogued and can be queried to check the progress of the bulk import task. This activity also shows the GET request you can use to check the progress of the bulk import task.

Workflow order of operations

To designate a csv file for bulk user import, the following tasks must be completed successfully:

  1. Make a POST request to the /environments/{environmentId}/importTasks service to create the new import task resource.

  2. Make a POST request to /environments/{environmentId}/importTasks/{taskId}/file to specify the csv file to use for the bulk user import.

  3. Make a GET request to /environments/{environmentId}/importTasks to return the full list of import task resources.

  4. Make a GET request to /environments/{environmentId}/importTasks/{taskId} to check the progress of the bulk user import task.

Note: The import tasks service requires the dir:import:user permission, which is associated with the p1:import:env:user scope. Administrators who perform bulk import tasks must have the p1:import:env:user scope encoded in their access token.

Step 1: Create the import task resource

You can create an import task resource to initiate an import operation. The request body for the POST must provide values for these properties:

  • emails

    Specifies the email address for the user, required for the notifications list.

  • users

    Specifies the following details required for the import of individual users:

    • passwords. Specifies the type of password encryption. Accepted values are BCRYPT and NONE. For more information about password encoding, see Set password.
    • state. Specifies the user state. Accepted values are ENBLED and DISABLED.
    • populationId. Specifies the populations resource to associate with the imported user.

The POST /environments/{environmentId}/importTasks operation creates the new import task resource.

curl -X POST "https://api.pingone.com/v1/environments/{environmentId}/importTasks" \
-H "Content-type: application/json" \
-H "Authorization: Bearer jwtToken"
-d "{
    "emails": "<emailAddress>",
    "users": {
        "passwords": "none",
        "state": "ENABLED",
        "population": {
            "id": "<populationId>"  
        }
    }
}"

The POST returns a 201 Created message. The response data for the new import task resource looks like this:

{
    "_links": {
        "self": {
            "href": "https://api.pingone.com/v1/environments/{environmentId}/importTasks/ab6b24ea-011c-4c70-98a8-df902236a58d"
        },
        "environment": {
            "href": "https://api.pingone.com/v1/environments/{environmentId}"
        }
    },
    "id": "ab6b24ea-011c-4c70-98a8-df902236a58d",
    "status": "PENDING",
    "users": {
        "passwords": "NONE",
        "state": "ENABLED",
        "population": {
            "id": "7a83c51e-e81f-429c-8961-0f56299878fd"
        }
    }
}

Step 2: Upload and process a file

The "id": "ab6b24ea-011c-4c70-98a8-df902236a58d" resource ID attribute returned in Step 1 is used in the request URL in Step 2 to specify the import task resource associated with the file upload. The administrator must initiate the file upload request using the task ID from Step 1 within five minutes of receiving the ID. If the administrator waits longer than five minutes, or if the task ID is used for another task, then a new task ID must be acquired.

To process the csv file, the request for this POST must include the following HTTP headers:

  • Content-Type: text/csv

    Specifies text (comma-separated values) as the input type for the file.

  • Content-Disposition: attachment; filename="<fileName>"

    Specifies an attachment, which identifies the file to be uploaded. The filename="filename.csv" parameter uses quoted string syntax.

  • Transfer-Encoding: chunked

    Divides the data stream into non-overlapping chunks. This parameter is required for any large file attachment. If omitted, the response returns a 413 Request entity too large error.

Note: A bulk user import task can process a maximum of 100,000 users or a file size of 200 MB. The Transfer-Encoding: chunked header is required for any bulk user import task, regardless of user count or file size.

The POST /environments/{environmentId}/importTasks/{taskId}/file operation uploads and processes the specified csv file.

curl -X POST "/environments/{environmentId}/importTasks/{taskId}/file" \
-H "Content-type: text/csv" \
-H "Content-Disposition: attachment; filename="<fileName>"" \
-H "Transfer-Encoding: chunked" \
-H "Authorization: Bearer jwtToken"

The response returns a 202 Accepted message and starts processing the task.

Step 3: Get all import tasks

The GET /environments/{environmentId}/importTasks operation provides information and HAL links to all defined import task resources. If you do not know the ID for an import task, you can get the list of all tasks to help you find the task that you want to check.

curl -X "GET" "https://api.pingone.com/v1/environments/{environmentId}/importTasks" \
-H 'Content-type: application/json' \
-H 'Authorization: Bearer jwtToken'

The response data looks like this:

{
    "_links": {
        "self": {
            "href": "https://api.pingone.com/v1/environments/{environmentId}/importTasks"
        },
        "environment": {
            "href": "https://api.pingone.com/v1/environments/{environmentId}"
        }
    },
    "_embedded": {
        "importTasks": [
            {
                "_links": {
                    "self": {
                        "href": "https://api.pingone.com/v1/environments/{environmentId}/importTasks/ab6b24ea-011c-4c70-98a8-df902236a58d"
                    },
                    "environment": {
                        "href": "https://api.pingone.com/v1/environments/{environmentId}"
                    },
                    "file": {
                        "href": "https://api.pingone.com/v1/environments/{environmentId}/importTasks/ab6b24ea-011c-4c70-98a8-df902236a58d/file"
                    }
                },
                "id": "ab6b24ea-011c-4c70-98a8-df902236a58d",
                "status": "PROCESSING",
                "users": {
                    "passwords": "NONE",
                    "state": "ENABLED",
                    "population": {
                        "id": "7a83c51e-e81f-429c-8961-0f56299878fd"
                    }
                },
                "file": {
                    "name": "5000_users_to_import.csv",
                    "length": "6.7kB",
                    "columns": 6
                },
                "results": {
                    "total": 5000,
                    "created": 4913,
                    "failures": 86,
                    "errors": [
                        {
 ...

Step 4: Check the task status

You can use the GET /environments/{environmentId}/importTasks/{taskId} operation to check the progress of the import task kicked off in Step 2. The request URL specifies the import task ID {taskId} generated in Step 1.

curl -X "GET" "https://api.pingone.com/v1/environments/{environmentId}/importTasks/ab6b24ea-011c-4c70-98a8-df902236a58d" \
-H 'Content-type: application/json' \
-H 'Authorization: Bearer jwtToken'

The response data (with the errors list removed) looks like this:

{
    "_links": {
        "self": {
            "href": "https://api.pingone.com/v1/environments/{environmentId}/importTasks/ab6b24ea-011c-4c70-98a8-df902236a58d"
        },
        "file": {
            "href": "https://api.pingone.com/v1/environments/{environmentId}/importTasks/ab6b24ea-011c-4c70-98a8-df902236a58d/file"
        },
        "environment": {
            "href": "https://api.pingone.com/v1/environments/{environmentId}"
        }
    },
    "id": "ab6b24ea-011c-4c70-98a8-df902236a58d",
    "status": "PROCESSING",
    "users": {
        "passwords": "NONE",
        "state": "ENABLED",
        "population": {
            "id": "7a83c51e-e81f-429c-8961-0f56299878fd"
        }
    },
    "file": {
        "name": "100000_big_users_to_import.csv",
        "length": "8.2kB",
        "columns": 6
    },
    "results": {
        "total": 100000,
        "created": 0,
        "failures": 0
    }
}

The status attribute shows the current status of the task. Possible values are: PENDING, PROCESSING, COMPLETE, and CANCELED. The results attribute shows the total number of users to be created by this task, the number of users created, and the number of failures, if any. Processing errors are identified by line number. The data looks like this:

"results": {
                    "total": 5000,
                    "created": 4913,
                    "failures": 86,
                    "errors": [
                        {
                            "line": 111,
                            "code": "INVALID_VALUE",
                            "target": "newPassword",
                            "message": "New password did not satisfy password policy requirements"
                        },
                        {
                            "line": 333,
                            "code": "INVALID_VALUE",
                            "target": "newPassword",
                            "message": "New password did not satisfy password policy requirements"
                        },
                        {
                            "line": 444,
                            "code": "INVALID_VALUE",
                            "target": "newPassword",
                            "message": "New password did not satisfy password policy requirements"
                        },
...

The errors attribute provides information about entries in the file that could not be processed. For example, for duplicate users errors, the code and message attributes in the response data identify the type of error:

"results": {
                    "total": 100,
                    "created": 0,
                    "failures": 100,
                    "errors": [
                        {
                            "line": 7,
                            "code": "UNIQUENESS_VIOLATION",
                            "target": "username",
                            "message": "A user with the specified username already exists."
                        },
                        {
                            "line": 1,
                            "code": "UNIQUENESS_VIOLATION",
                            "target": "username",
                            "message": "A user with the specified username already exists."
                        },
                        {
                            "line": 9,
                            "code": "UNIQUENESS_VIOLATION",
                            "target": "username",
                            "message": "A user with the specified username already exists."
                        },
...