Nuxeo digest linker
By nuxeo
This Nuxeo add-on is designed to establish links between existing binary files and Nuxeo documents through REST API calls like create document or update document. So client can provide a known md5 digest for a file already stored in a storage bucket to tie it to a target document.
nuxeo-digest-linker
Overview
Description:
This Nuxeo add-on is designed to establish links between existing binary files and Nuxeo documents through REST API calls.
json
{
"properties": {
"file:content": {
"validate": "true",
"name": "pdf",
"mime-type": "application/pdf",
"digest": "c1f4d6d02bacff4b61332c239ec06bfe$",
"length": -1
}
}
}
Binary Storage:
The binaries are stored in a storage location identified by their MD5 hash.
Prerequisite:
Assuming that the binary files have already been uploaded to a cloud storage service, such as an S3 bucket, this add-on can assist in linking document metadata with the binary files using their MD5 digest.
Linking Procedure:
The link to the binary files can be incorporated within the JSON structure of the document during both document creation and update requests in Nuxeo.
Digest Submission:
The MD5 digest is to be submitted within the file:content block.
Security and permissions:
User must be admin to do the blob linking.
If you want to grant other users access to this feature there two choices.
As a rule of thumb permission of which user can and which user can not use this feature is determined of the current Blob Provider what could be different depending what object storage is used.
Cloud deployment
If S3BlobProvider or similar manager is used then it already has functionality inherited from class "AbstractBlobProvider".
In this case you can configure blob provider to grant access to users or groups by setting two config properties on provider configuation
```xml
User1,User2 Group1,Group2 ```
Custom solution for local deployment and special blob manager
Alternatively, if you are using a custom blob manager, then you have to implement method hasCreateFromKeyPermission as shown in example bellow
```xml <?xml version="1.0"?>
org.nuxeo.ecm.core.test.blobmanager nuxeo.digest.linker.TestBlobProvider
```
Java code
```java /******************************* * @return true of user is admin *******************************/ @Override public boolean hasCreateFromKeyPermission() { NuxeoPrincipal principal = NuxeoPrincipal.getCurrent();
if (null == principal) {
log.debug("No principal found");
return false;
}
String name = principal.getName();
Boolean isAdmin = principal.isAdministrator();
String digest = JSONDigestBlobDecoder.currentDigest();
boolean res = isAdmin;
//TODO: Optionally put some additional logic to validate digest/principal permissions for non-admin user
log.debug("For digest [{}] and principal [{}] (admin={}), returning=[{}]", digest, name, isAdmin, res);
return res;
}
```
Add-On Functionality:
The add-on checks for the presence of a digest element (see examples provided), and if it is found, it associates the corresponding existing binary file from the default binary storage with the document.
Installation
Download package from the marketplace for manual installation
or install directly from the marketplace
shell
bin/nuxeoctl mp-install nuxeo-digest-linker
Usage
- The JSON below illustrates a sample usage of the addon. This addon allows the utilization of the binary's digest within a blob schema.
json
{
"file:content": {
"name": "random_text_2.pdf",
"digest": "7081efb5c2db98769b4df99c66f0cd2f"
}
}
It functions correctly as long as a binary with the specified digest is present in the default binary storage. By default, the addon does not generate an error if the binary is missing, but in such cases, it cannot calculate the file's length or generate its thumbnail.
To use a different binary provider, include a
providerId
value in the structure below.
json
{
"file:content": {
"name": "random_text_2.pdf",
"digest": "7081efb5c2db98769b4df99c66f0cd2f",
"providerId": "default"
}
}
- If you want the system to throw an error when the digest refers to a non-existing blob,
add the parameter
"validate": "force"
.
json
{
"file:content": {
"name": "random_text_2.pdf",
"digest": "7081efb5c2db98769b4df99c66f0cd2f",
"validate": "force"
}
}
When the length attribute is provided and its value is not equal to -1, it will be utilized to specify the file size. This optimization significantly enhances performance.
json
{
"file:content": {
"name": "random_text_2.pdf",
"digest": "7081efb5c2db98769b4df99c66f0cd2f",
"length": "174315234"
}
}
Log config
Add line to lib/log4j2.xml
xml
<Logger name="org.nuxeo.digestlinker" level="debug" />
S3 testing using minio
shell
bin/nuxeoctl mp-install amazon-s3-online-storage
Install minio (need linux VM or docker)
Create bucket in minio
set config properties
```properties nuxeo.templates=...,s3binaries nuxeo.s3storage.bucket=nuxeo nuxeo.s3storage.awsid=minioadmin nuxeo.s3storage.awssecret=minioadmin nuxeo.s3storage.region=us-west-1
path to minio installation
nuxeo.s3storage.endpoint=http://pi:9000 nuxeo.s3storage.isLocal=true nuxeo.s3storage.pathstyleaccess=true nuxeo.s3storage.credsType=BASIC nuxeo.s3storage.directdownload=true nuxeo.s3storage.accelerateMode=false ```
Azure testing with azurite
To test in azure dev mode
Azure CLI first
brew install azure-cli
Azure cross-platform Azure Storage emulator
npm install -g azurite
Start azurite for blob storage
export NUXEOHOME=. && azurite-blob --silent --location $NUXEOHOM/blob/storage --debug $NUXEO_HOME/logs/azure.log
List containers
az storage container list
Create container
az storage container create --name nuxeo
List blobs
az storage blob list --container-name nuxeo
Configure nuxeo to use azurite
shell
bin/nuxeoctl mp-install microsoft-azure-online-storage
add these to nuxeo.conf
properties
nuxeo.templates=...,azurebinaries
nuxeo.core.binarymanager=org.nuxeo.digestlinker.azure.DevModeAzureBinaryManage
nuxeo.storage.azure.container=nuxeo
Contributing
Hyland developers are encouraged to contribute to the development of this addon to enhance its functionality and performance.
License
LTS 2023 | LTS 2021 | LTS 2019 | LTS 2017 | |
---|---|---|---|---|
2024.2.2908 | ||||
2024.1.2601 | ||||
2023.11.2207 | ||||
2023.11.2204 | ||||
2023.0.0-11-2204 |
Deprecated | Target platforms Name | Target platforms Range | Target platforms Pattern | Updated | |
---|---|---|---|---|---|
2024.2.2908 | - | lts | [2021.0,) | - | Mar 1, 2024, 3:20:52 PM |
2024.1.2601 | - | lts | [2021.0,) | - | Jan 26, 2024, 5:51:44 PM |
2024.1.2601-SNAPSHOT | - | lts | [2021.0,) | - | Jan 26, 2024, 3:40:44 PM |
2023.11.2207 | - | lts | [2021.0,) | - | Jan 26, 2024, 3:38:55 PM |
2023.11.2204 | - | lts | [2021.0,) | - | Nov 22, 2023, 5:14:35 PM |
2023.0.0-11-2204 | - | lts | [2021.0,) | - | Nov 22, 2023, 5:05:20 PM |