AWS Snowball - Data Migration at Scale

The AWS Snowball Edge service provides Snowball devices for large-scale data transfer and data migration. Snowball offers rugged, secure options to move terabytes or petabytes of data to AWS.

Snowball uses encryption to protect data and integrates with AWS services like Amazon S3 and Amazon EC2. It is managed through the AWS console or AWS CLI.

AWS Snowball Edge options

The Snowball Edge comes in multiple configurations to support different use cases:

• Snowball Edge Storage Optimized
  • Up to 80TB of usable storage
  • Optimal for data transfer and migration
  • Suited for large datasets, media files, IoT sensor data
• Snowball Edge Compute Optimized
  • 42 vCPUs, 80 GiB of memory
  • Optional NVIDIA Tesla V100 GPU
  • Runs EC2 instances and Lambda functions
  • For compute-intensive workloads at edge locations
• AWS Snowcone
  • Small 8TB data transfer device
  • Highly portable
  • Useful for collecting data in remote locations

Key features:

• All Snowball Edge options use ruggedized enclosures designed for transport
• Built-in LCD display to manage device and view status
• 256-bit encryption to secure data
• High-speed 100 Gbps network connectors
• Designed for use in locations with intermittent connectivity

Data transfer:

• Import/export between on-premises data stores and Amazon S3
• Use file interfaces or Amazon S3 adapter to transfer data
• Bypass internet limitations for large datasets

Edge computing:

• Run EC2 instances and Lambda functions on Snowball Edge
• Execute machine learning inferencing at edge locations
• Operate Kubernetes clusters with Amazon EKS Anywhere

Management:

• Manage devices using AWS console, AWS CLI, or AWS OpsHub
• Monitor status, unlock devices, transfer files
• Seamless integration with AWS services

The Snowball Edge options provide flexible capabilities to transfer data at scale and perform compute at the edge.

Data transfer capabilities

Snowball Edge provides high-speed data transfer capabilities:

• Imports/exports between AWS and on-premises environments
• Bypasses internet bottlenecks or limits

Optimized for large data transfers:

• Up to 80TB per Snowball Edge device
• Additional devices can scale capacity
• Alternative to slow internet transfers

High transfer speeds:

• 100 Gbps network connectors
• Fast edge locational transfers
• Faster than internet limitations

Rugged for transport:

• Tough enclosures to protect data
• Built-in carriers and shipping
• Track shipments in AWS console

Secure transfer:

• 256-bit encryption secures data in transit
• Secure erase after AWS data import

Edge location usage:

• Keep data local for low latency access
• Operate on data before transfer
• Move large data sets from remote sites

Seamless AWS integration:

• Import into Amazon S3 buckets
• Export S3 data to Snowball Edge
• Leverage services like AWS Lambda

With Snowball Edge, users can optimize and accelerate large data transfers, leverage edge locations, and integrate with AWS services for data portability.

Security features

Snowball Edge devices provide robust security:

• 256-bit encryption to secure data
• Tamper-resistant enclosures
• Secure data erase after AWS import

Encryption

• 256-bit encryption protects data at rest
• Data encrypted/decrypted seamlessly
• Keys managed securely by AWS

Secured enclosures

• Rugged, tamper-resistant physical case
• Protects against data theft
• Evident if compromised

Shipping security

• Tracked shipping with notifications
• Automatic return shipping label
• Status updates in AWS console

AWS integration

• Seamless identity and access control with AWS IAM
• Leverage AWS security services

On-device security

• Unlock and authenticate devices before use
• Protocols like SSH for secure connections
• Use AWS key pairs for access control

Compliance

• Adheres to strict AWS security standards
• Meets compliance requirements (HIPAA, PCI, etc)

With multiple encryption, hardware, and access control security measures, Snowball Edge keeps data ultra-secure throughout the process.

AWS Snowball Management

Snowball Edge provides multiple methods to easily manage devices and data:

AWS Console

• Create and manage Snowball jobs
• Monitor status and track shipments
• View usage reports and logs

AWS CLI

• Automate Snowball tasks with CLI
• Transfer files and data sets
• Script management processes

AWS OpsHub

• Unlock and configure devices
• Visual dashboard to manage Snowballs
• Transfer files through graphical interface

On-device capabilities

• Built-in LCD display to view status
• Provides device IP address for access
• Reboot, shutdown, cluster devices

Key management tasks:

• Order and configure Snowball devices
• Connect and unlock Snowballs
• Transfer data to/from devices
• Monitor shipment and job status
• Shut down and return devices when complete

Integrations:

• Utilize Amazon S3 adapter for transfers
• Manage Amazon EC2 instances on devices
• Run and monitor AWS Lambda functions

Automation:

• Script management with AWS CLI
• Trigger actions based on events
• Streamline at scale with many devices

With integrated console, CLI, and on-device management, Snowball Edge enables users to easily control devices and data.

Integration with AWS services

Snowball Edge provides tight integration with AWS services:

Amazon S3

• Import data into S3 buckets
• Export S3 data to Snowball devices
• Use S3 adapter for programmatic transfers

Amazon EC2

• Launch EC2 instances on Snowball Edge
• Run compute workloads at edge locations
• Leverage EC2 functionality locally

AWS Lambda

• Execute Lambda functions on device
• Trigger Lambdas based on device events
• Enable serverless computing on edge

AWS IoT Greengrass

• Run AWS Greengrass locally on devices
• Perform inference and analytics on device
• Operate machine learning at edge

Amazon EKS Anywhere

• Deploy Kubernetes clusters on devices
• Run containerized apps at edge locations
• Simplified management with EKS Anywhere

Other integrations

• AWS Identity and Access Management (IAM)
• Amazon Elastic Block Store (EBS)
• AWS Security Token Service (STS)
• AWS Systems Manager

Benefits:

• Seamlessly leverage AWS services locally
• Reduce data transfer costs and latency
• Expand AWS capabilities to edge locations
• Streamline management and security with AWS

With deep AWS service integrations, Snowball Edge extends AWS functionality to edge locations and on-prem infrastructures.

Ordering and setup

Snowball Edge provides a streamlined process to get started:

Ordering

• Initiate jobs in AWS console
• Select device type and size
• Choose shipping preferences

Delivery

• Devices delivered in 1-2 days
• Track shipments in console

Setup

• Connect device to network
• Unlock with credentials
• Authenticate and configure

Transfer data

• Use file interfaces or Amazon S3
• CLI automation and scripts
• AWS OpsHub graphical transfers

Return shipment

• Snowball provides return label
• Track return status in console

AWS import

• Snowball ships data to AWS
• Ingest and verification
• Notification upon completion

Monitoring

• Track status on LCD display
• View logs and metrics
• Monitor ingest in console

Security

• Transfer data securely (encryption)
• Protect devices physically
• Securely erase data after ingest

With integrated shipping, tracking, and data transfers, Snowball Edge simplifies large-scale data migrations.

Storage and compute capacities

Snowball Edge provides robust storage capacity and compute capacity:

Storage options

• Snowball Edge Storage Optimized
  • 80TB capacity
  • Data transfer focused
• Snowball Edge Compute Optimized
  • 42TB usable storage
  • Balance of storage and compute
• AWS Snowcone
  • 8TB capacity
  • Small form factor

Scalable storage

• Cluster devices for larger storage pools
• Grow/shrink storage as needed
• On-device object storage

Compute options

• Up to 52 vCPUs per device
• Up to 208 vCPUs clustered
• Optional NVIDIA Tesla V100 GPU
• sbe1, sbe-c, sbe-g instance types

Local compute

• Run EC2 instances and AWS Lambda locally
• Execute machine learning inference
• Operate Kubernetes clusters

High speed processing

• Fast local data access
• Avoid round-trip data transfers
• Leverage edge processing power

Integrated data and compute

• Collocated storage and analysis
• Optimize workflows with data locality
• Securely operate on sensitive data

With robust storage scalability and compute capacity, Snowball Edge is ideal for large-scale edge data and analytics.

Use cases

Common use cases for Snowball Edge include:

Data migration

• Migrate data centers to AWS
• Consolidate data silos to the cloud
• Backup archives, databases, repositories

Data analytics

• Operational analytics at edge locations
• Machine learning inference on datasets
• Analyze locally before cloud transfer

Media processing

• Transfer large video files
• Render clips before cloud upload
• Encode and transcode media libraries

Genomics and research

• Manage genomic sequencer outputs
• Share large datasets across sites
• Collaborate on data analysis

IoT and industrial

• Collect and process sensor data
• Monitor equipment and diagnose issues
• Optimize manufacturing and operations

Hybrid cloud

• Replicate data bi-directionally
• Cloud bursting for capacity expansion
• Transition between cloud and on-prem

Compliance

• Meet internal policies for controlled data
• Satisfy regulatory requirements
• Manage sensitive data securely

With robust capabilities and integrations, Snowball Edge powers many workloads at the edge and in the cloud.

Conclusion

AWS Snowball Edge provides a powerful service to transfer large datasets and perform compute at the edge. Key benefits include:

• Accelerating data transfers with portable devices
• Enabling edge computing for low latency
• Integrating tightly with AWS services
• Securing data with encryption and access controls
• Simplifying large-scale data migration
• Avoiding network bottlenecks or limits
• Leveraging AWS’s proven infrastructure
• Scaling capacity with multiple Snowball devices
• Streamlining management with unified interfaces

With its robust capabilities, security, and seamless integration, Snowball Edge unlocks the potential of the edge and the cloud.