File upload

Overview

This set of RESTful API designed for managing file operations, including upload, download, and status checking and also searching the STAC layers. It is part of a larger system that handles file ingestion and tracking. The API is secured and requires appropriate permissions for access. All the APIs are secured and need API-Key to access them.

Please refer to the slide gIQ-Authentication for detailed instructions on how to generate the API key.

---

1. File Upload API

There are two methods for uploading files. For smaller files (up to 100MB), you can use Type 1, which allows you to upload the file directly. For larger files, it's recommended to use Type 2, where the file is uploaded in chunks and then combined. Detailed steps for both methods are provided in their respective sections.

1.1 Type 1 (Up to 100MB)

Endpoint

• URL: https://giq.ae/public/file/upload
• Method: POST
• Produces: application/json

Description

This API allows users to upload a file for processing. File Upload Type 1 API is designed for uploading smalled files ( up to 100MB or lesser).

Permissions Required

At least one of the following permissions is required:

• files_admin
• files_upload

Request Parameters

Name	Type	Mandatory	Description
file	MultipartFile	Yes	The file to be uploaded.
sourceType	String	No	The type of source from predefined source types. Default is UNKNOWN. Including it ensures more accurate file processing.

Possible Source Types

Possible Values
MULTISPECTRAL	POINT	GEOMETRYCOLLECTION	NETCDF
HYPERSPECTRAL	LINESTRING	TILES3D	LAND_SUBSIDENCE
SAR	POLYGON	COMPLEX	PDF
FULL_MOTION_VIDEO	HEIGHTMAP	UNKNOWN	IMAGE
PANCHROMATIC	THERMAL	ANNOTATION	CZML

Request Example (cURL)

curl --location 'https://giq.ae/public/file/upload?sourceType=IMAGE' \
--header 'X-API-KEY: apiKey' \
--form 'file=@"/Users/Downloads/test.png"'

Success Response

Field	Type	Description
fileId	String	Unique identifier for the uploaded file.
originalName	String	The original name of the uploaded file.
createdAt	Date	The date and time when the file was uploaded.
size	Long	The size of the uploaded file in bytes.

Success Response Example

{
  "fileId": "UUID",
  "originalName": "test.png",
  "createdAt": "2024-09-17T10:30:00Z",
  "size": 2048
}

Error Handling

Status Code	Error Message	Description
401 Unauthorized	Unauthorized Request	Indicates invalid credentials.
400 Bad Request	Bad Request	Indicates an invalid request, such as a missing file.
403 Forbidden	Permission Denied	Occurs when the user lacks the required permissions (files_admin or files_upload).
500 Internal Server Error	Internal Server Error	Indicates a server error occurred while processing the request.

---

1.2 Type 2

Overview

File Upload Type 2 API is designed for uploading large files (up to ~10GB or larger) in chunks to ensure smooth uploads, especially when dealing with potential network interruptions. Chunked file uploads offer the following benefits:

• Reduces the risk of failure due to network issues or timeouts.
• Allows the ability to resume incomplete uploads.
• Improves the reliability of uploading large files.
• Optimizes bandwidth by retrying specific parts if needed.
• Allows users the flexibility to upload chunks in sizes they define.

This API consists of three main steps:

1. Generate an Upload ID – Initiates the multipart upload process.
2. Upload File in Chunks – Uploads individual file chunks.
3. Complete the Upload – Finalizes the upload process once all chunks are uploaded.

---

Step 1: Generate Upload ID

Endpoint

• URL: https://giq.ae/public/file/generateUploadId
• Method: POST
• Produces: application/json

Description

Generates an Upload ID required for multipart file uploads. This Upload ID will be used in subsequent API calls for uploading file chunks and completing the upload.

Permissions Required

At least one of the following permissions is required:

• files_admin
• files_upload

Request Parameters

Name	Type	Required	Description
fileName	String	Yes	The name of the file to be uploaded.

Success Response

A JSON response containing the Upload ID and the file path (key) for the upload.

Success Response Example

{
  "uploadId": "abcd1234",
  "key": "path/to/your/file.extension"
}

Error Handling

Status Code	Error Message	Description
500 Internal Server Error	Failed to generate upload ID	Indicates a server error during the upload initiation process.

---

Step 2: Upload Chunk

Endpoint

• URL: https://giq.ae/public/file/uploadChunk
• Method: POST
• Produces: application/json

Description

Uploads a chunk of the file as part of a multipart upload. This step is repeated for each chunk until the entire file is uploaded.

Request Parameters

Name	Type	Required	Description
file	MultipartFile	Yes	The chunk of the file to be uploaded.
uploadId	String	Yes	The unique ID generated from the /generateUploadId API.
partNumber	Integer	Yes	The part number for the chunk. Each chunk must have a unique part number.
key	String	Yes	The file path (key) for the upload.

Success Response

A message confirming the chunk was uploaded successfully.

Success Response Example

{
  "message": "Chunk uploaded successfully"
}

Error Handling

Status Code	Error Message	Description
500 Internal Server Error	Failed to upload chunk	Indicates a server error while uploading the file chunk.

---

Step 3: Complete Upload

Endpoint

• URL: https://giq.ae/public/file/completeUpload
• Method: POST
• Produces: application/json

Description

This API completes the multipart upload process by assembling the uploaded file chunks into the final file and starts processing it.

Permissions Required

At least one of the following permissions is required:

• files_admin
• files_upload

Request Body

Name	Type	Required	Description
uploadId	String	Yes	The unique ID generated from the /generateUploadId API.
key	String	Yes	The file path (key) generated from the /generateUploadId API.
fileSize	Long	Yes	The total size of the uploaded file in bytes.
sourceType	SourceType	Yes	The source type of the file (refer to the enum above).
vectorMappingProperties	JsonNode	No	Optional vector mapping properties for the file.
layerProperties	JsonNode	No	Optional layer-specific properties.
geometry	String	No	Optional geometry information related to the file.
acquisitionDate	Date	No	Optional acquisition date of the file.

Success Response

Field	Type	Description
fileId	String	Unique identifier for the uploaded file.
originalName	String	The original name of the uploaded file.
createdAt	Date	The date and time when the file was uploaded.
size	Long	The total size of the uploaded file in bytes.

Success Response Example

{
  "fileId": "file1234",
  "originalName": "example.png",
  "createdAt": "2024-09-17T10:30:00Z",
  "size": 10485760
}

Error Handling

Status Code	Error Message	Description
500 Internal Server Error	Failed to complete upload	Indicates a server error during the completion process.

Notes

• Ensure that the uploadId and key used in the request match the values generated during the upload initiation.

• The system attempts to clean up any partially uploaded chunks in case of failure during the completion process.

• If vector mapping or layer properties are relevant, they can be added to the request for more detailed file processing.

2. Get File Status

Overview

This API allows users to retrieve the status of a file by its unique identifier.

Endpoint

• URL: https://giq.ae/public/file/status/{id}
• Method: GET
• Produces: application/json

Description

Retrieves the current status and metadata of a file based on its unique file ID.

Permissions Required

At least the following permission is required:

• files_admin

Request Path Variables

Name	Type	Required	Description
id	String	Yes	The unique identifier of the file which was generated by upload file API.

Request Example (cURL)

curl --location 'https://giq.ae/public/file/status/e098356c-bf59-11ee-a152-79143a5520da' \
--header 'X-API-KEY: apiKey'

---

Response

Response Fields

Field	Type	Description
id	String	Unique identifier of the file.
originalName	String	The original name of the uploaded file.
status	String	The current status of the file.
createdAt	Long	The date and time when the file was created (in epoch milliseconds).

Sample Response

{
  "id": "e098356c-bf59-11ee-a152-79143a5520da",
  "originalName": "test.png",
  "status": "COMPLETED",
  "createdAt": 1706610323613
}

---

Error Handling

If an error occurs, appropriate status codes and error messages are returned.

Status Code	Error Message	Description
404 Not Found	File not found	Indicates that the file with the given id does not exist.
500 Internal Server Error	Internal Server Error	Indicates a server error occurred while processing the request.

Error Response Example

{
  "error": "File not found"
}

---

Notes

• Ensure that the correct fileId is provided in the request to avoid a "File not found" error.
• The status field reflects the current status of the file, such as QUEUED, IN_PROGRESS,COMPLETED or FAILED.

3. Download File URL

Overview

This API generates a pre-signed URL for downloading a file. The URL allows temporary access to download the file securely. Please note URL will be expired after 24 hours.

---

Endpoint

• URL: https://giq.ae/public/file/download/url
• Method: GET
• Produces: text/plain

Description

Generates a pre-signed URL for downloading a file identified by its fileId. The user can optionally specify whether to use the default content type for the file.

Permissions Required

At least the following permission is required:

• files_admin

Request Parameters

Name	Type	Required	Description
fileId	String	Yes	The unique identifier of the file( Which was generated by upload file api) to be downloaded.
useDefaultContentType	Boolean	No	Optional. If set to true, the file will be downloaded with the default content type. Default is false.

Request Example (cURL)

curl --location 'https://giq.ae/public/file/download/url?fileId=e098356c-bf59-11ee-a152-79143a5520da&useDefaultContentType=true' \
--header 'X-API-KEY: apiKey'

---

Response

The response will return a pre-signed URL in plain text format, which can be used to download the file.

Error Handling

If an error occurs, the API returns appropriate status codes and error messages.

Status Code	Error Message	Description
500 Internal Server Error	Internal Server Error	Indicates a server error occurred while processing the download request.

---

Notes

• The pre-signed URL is temporary and will expire after 24Hours.
• Ensure that the fileId provided is correct to avoid download issues.
• The useDefaultContentType parameter is optional. If not provided, the default behavior is to return the file without enforcing a content type.