Skip to main content

Connector Core Concepts

Understanding the different types of connectors and how they work is essential for effective data integration.

Connector Types​

NeoSpace supports multiple connector types for different data sources:

S3-Compatible Connectors​

S3-compatible connectors connect to S3-compatible object storage systems.

Supported Systems:

  • AWS S3
  • MinIO
  • Other S3-compatible storage systems

Features:

  • Bucket and path configuration
  • Access key and secret key authentication
  • Support for various file formats (CSV, Parquet, JSON)
  • Incremental file sync

Configuration:

  • Endpoint: S3-compatible endpoint URL
  • Bucket: S3 bucket name
  • Access Key: AWS access key ID
  • Secret Key: AWS secret access key
  • Region: AWS region (if applicable)

Oracle Connectors​

Oracle connectors provide direct connection to Oracle databases.

Features:

  • Direct database connection
  • SQL query support
  • Table and view access
  • Automatic schema detection

Configuration:

  • Host: Oracle database host
  • Port: Database port (default 1521)
  • Database: Database name
  • Username: Database username
  • Password: Database password
  • Service Name: Oracle service name

Oracle S3-Compatible Connectors​

Oracle S3-compatible connectors combine Oracle database access with S3-compatible storage.

Features:

  • Access to Oracle database
  • Integration with S3-compatible storage
  • Hybrid data access patterns
  • Flexible data source configuration

Use Cases:

  • Oracle databases with S3 backup storage
  • Hybrid data architectures
  • Data migration scenarios

Integration Architecture​

Connectors follow a consistent architecture:

Architecture Components:

  • Authentication Layer: Handles credential management and authentication
  • Connection Layer: Manages connections to data sources
  • Schema Detection: Automatically detects data schemas
  • Data Access Layer: Provides access to data files and tables
  • Validation Layer: Validates data quality and format

Architecture Benefits:

  • Unified Interface: Consistent interface across all connectors
  • Automatic Processing: Automatic schema detection and validation
  • Secure Access: Secure credential storage and transmission
  • Error Handling: Robust error handling and retry logic

Next Steps​