Follow

Figure Eight Secure Data Access - AWS Integration

 

When utilizing Figure Eight’s Secure Data Access, your team maintains ownership and governance overall source data. The data that your team supplies never leave your servers. For added security, private buckets can be used while processing training data in the Figure Eight platform.

Your team serves the source data via secure URLs hosted in private buckets inside your cloud storage. The only data that is passed to Figure Eight are the URLs for your private bucket, which will be assigned a unit ID. Corresponding annotations for the data can be downloaded from the Figure Eight platform and can subsequently be associated with source data via the unit ID.

  • Secure content is rendered through signed URLs
  • Signed URLs expire immediately after the content is rendered
  • Your content is never stored or saved within the Figure Eight platform
  • Content is rendered only to authenticated contributors and requestors with access to specific Figure Eight tasks. 

Note: For access to this feature, please contact your Customer Success Manager or Account Executive.

S3 Bucket Preparation

  • Within S3, create a new bucket or locate an existing bucket.
    • For image annotation, pixel-level semantic segmentation, and text annotation uses cases, the S3 bucket will need to be CORS configured.

Screen_Shot_2019-10-18_at_3.40.07_PM.png

Figure 1. Create new/locate existing S3 bucket

Create IAM Policy

  • Select "Policies" on the left and "Create policy".

Screen_Shot_2019-10-18_at_3.41.25_PM.png

Figure 2. Create policy

Input JSON

  • Within the JSON editor, copy, paste, and modify the JSON below:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AllowReadOnlyOperations",
"Effect": "Allow",
"Action": [
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::s3BucketName",
"arn:aws:s3:::s3BucketName/*"
]
}
]
}

Screen_Shot_2019-10-18_at_3.43.32_PM.png

Figure 3. JSON Policy

  • Name this policy to reference later when creating the IAM role.
  • Click on "Create policy" when complete.

Create IAM Role

  • Back on the main page of IAM, select "Roles" on the left and “Create role”.
  • Select AWS service as type of trusted entity, S3 as the service that will use this role, and S3 as the use case.
  • Move on to “Permissions” when complete.

Screen_Shot_2019-10-18_at_3.49.21_PM.png

Figure 4. Create role

Link IAM Role to IAM Policy

  • Under "Attach permissions policies", find the IAM policy name created in the previous step.

Screen_Shot_2019-10-18_at_3.50.27_PM.png

Figure 5. Attach permissions policies

  • Name the IAM role, which will be referenced in the Figure Eight platform.

Screen_Shot_2019-10-18_at_3.51.03_PM.png

Figure 6. Name role

  • 3. Select the IAM role and copy the Role ARN (AWS Resource Name).
    • Note where the "Trust relationships" tab is as further configuration will be required at a later step.

1.png

Figure 7. Role Summary

Data & Security on Figure Eight

  • On the Figure Eight platform, navigate to your Account Page --> Data & Security Tab.
    • If you cannot find the Data & Security tab, please reach out to your Figure Eight Customer Success Manager or Support team.

Screen_Shot_2019-04-15_at_10.34.48_AM.png

Figure 8. Data & Security tab

  • Select "Create New Secure Storage"
  • Input Storage CML Name, which will be used in the job's CML on the Design Page.
    • There is a 10-character maximum limit for the Storage CML Name (only alphanumeric characters and underscores will be accepted).

Screen_Shot_2019-10-20_at_8.53.47_AM.png

Figure 9. Add New Storage

  • Paste your IAM Role ARN and select "AWS Region Name".
  • Select "Create".
  • Now, on the Data & Security Page, you will see a status "Pending" under Secure Storage.
    • Leave the Data & Security page open, as the Figure Eight External ID and Figure Eight account ID will be referenced in the next step.

Screen_Shot_2019-10-20_at_8.53.52_AM.png

Figure 10. Pending Figure Eight Secure Storage

Update the IAM Role Trust Relationship 

  • In the "Trust relationships" tab, select "Edit trust relationship".

Screen_Shot_2019-10-20_at_8.55.10_AM.png

Fig 11. Edit Trust Relationship

  • Within the Policy Document editor, input the following JSON:
    • Modify the JSON with your respective Figure Eight Account ID and Figure Eight External ID.
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Principal": {
            "Service": "s3.amazonaws.com"
          },
         "Action": "sts:AssumeRole"
      },
      {
         "Effect": "Allow",
         "Principal": {
            "AWS": "arn:aws:iam::
figureEightAccountId:root"
         },
         "Action": "sts:AssumeRole",
         "Condition": {
            "StringEquals": {
               "sts:ExternalId": "
figureEightExternalId"
            }
         }
      }
   ]

Screen_Shot_2019-10-20_at_8.55.42_AM.png

Figure 12. Edit Trust Relationship JSON

  • Under "Trusted entities", you should see the Figure Eight Account ID and under "Conditions" you should see the Figure Eight External ID.

Notify Figure Eight

  • Notify your Customer Success Manager that the setup is complete.
  • At this point, the Figure Eight DevOps team will set the Secure Storage to 'Active' in the backend.

Screen_Shot_2019-10-20_at_8.56.09_AM.png

Figure 13. Active Figure Eight Secure Storage

Upload Data with Secure Data Access Links

  • To use SDA hosted links, upload a CSV or URLs in the following format: 
    • s3://s3BucketName/bucketFilePath/fileName.fileType

Finishing Touch in CML

  • As a final step, navigate to your job's Design Page and update your column references in liquid with the following format:
    • {{ columnName | secure: 'storageCmlName' }}
    • When using videos with Secure Data Access make sure to include the following tag in the CML section of your job: preload="auto".
  • For confirmation, you should see your hosted data within the Preview Page but not outside of the Figure Eight platform. 

Additional Instructions: 

  • You can update or delete an existing storage integration
    1. Please note that changing the "Storage CML name" for a store provider will prevent any existing jobs that use that name in the CML from accessing private S3 content. So if value of "Storage CML Name" is modified the new value should be updated in the jobs accessing private data. 
    2. Modifying AWS Resource Name or AWS Region Name will break the existing integration. 
  • The team can set up multiple storage provider integrations
  • Team admins can add storage provider integration for their teams
  • Org admins can add storage provider integration for all the teams within the Org

Was this article helpful?
0 out of 0 found this helpful


Have more questions? Submit a request
Powered by Zendesk