a2252fa923
- Remove old Confluence variables - Add NEXT_PUBLIC_API_URL for API access 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
84 lines
1.9 KiB
Markdown
84 lines
1.9 KiB
Markdown
# Data Migration Scripts
|
|
|
|
This directory contains scripts for migrating filament data from Confluence to DynamoDB.
|
|
|
|
## Prerequisites
|
|
|
|
1. AWS credentials configured (either via AWS CLI or environment variables)
|
|
2. DynamoDB table created via Terraform
|
|
3. Confluence API credentials (if migrating from Confluence)
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
cd scripts
|
|
npm install
|
|
```
|
|
|
|
## Configuration
|
|
|
|
Create a `.env.local` file in the project root with:
|
|
|
|
```env
|
|
# AWS Configuration
|
|
AWS_REGION=eu-central-1
|
|
DYNAMODB_TABLE_NAME=filamenteka-filaments
|
|
|
|
# Confluence Configuration (optional)
|
|
CONFLUENCE_API_URL=https://your-domain.atlassian.net
|
|
CONFLUENCE_TOKEN=your-email:your-api-token
|
|
CONFLUENCE_PAGE_ID=your-page-id
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Migrate from local data (data.json)
|
|
|
|
```bash
|
|
npm run migrate
|
|
```
|
|
|
|
### Clear existing data and migrate
|
|
|
|
```bash
|
|
npm run migrate:clear
|
|
```
|
|
|
|
### Manual execution
|
|
|
|
```bash
|
|
# Migrate without clearing
|
|
node migrate-with-parser.js
|
|
|
|
# Clear existing data first
|
|
node migrate-with-parser.js --clear
|
|
```
|
|
|
|
## What the script does
|
|
|
|
1. **Checks for Confluence credentials**
|
|
- If found: Fetches data from Confluence page
|
|
- If not found: Uses local `public/data.json` file
|
|
|
|
2. **Parses the data**
|
|
- Extracts filament information from HTML table (Confluence)
|
|
- Or reads JSON directly (local file)
|
|
|
|
3. **Prepares data for DynamoDB**
|
|
- Generates unique IDs for each filament
|
|
- Adds timestamps (createdAt, updatedAt)
|
|
|
|
4. **Writes to DynamoDB**
|
|
- Writes in batches of 25 items (DynamoDB limit)
|
|
- Shows progress during migration
|
|
|
|
5. **Verifies the migration**
|
|
- Counts total items in DynamoDB
|
|
- Shows a sample item for verification
|
|
|
|
## Troubleshooting
|
|
|
|
- **Table not found**: Make sure you've run `terraform apply` first
|
|
- **Access denied**: Check your AWS credentials and permissions
|
|
- **Confluence errors**: Verify your API token and page ID
|
|
- **Empty migration**: Check that the Confluence page has a table with the expected format |