Google's Appengine relays all its data on the datastore, a big table. The datastore means all data, files, everything that can be stored. Do that, if we make a datastore's backup or a copy, we are going to be able to recreate the instance anywhere. This is very useful for backup/restore and for interchange data between environments like quality / testing / production servers.
In this post, we are going to show you how to make a backup and restore it. For that purpose we are going to need the bulk loader tool, included with our appengine's SDK.
The bulk loader tool uses the Google's Appengine remote API for access datastore remotely. For enabling this feature, we must enable the url in the app.yaml:
- url: /remote_api
script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
login: admin
Vikuit's code has this feature enabled by default. The next steps are really easy. If we want to download the data, we are going to use the follow the command:
[APP_ENGINE_SDK_FOLDER]>appcfg.py download_data --application=[APP_ID] --url=http://[APP_ID].appspot.com/remote_api --filename=data.exp
Where [APP_ENGINE_SDK_FOLDER] is your Google's Appengine SDK installation folder and [APP_ID] your application identifier.
If you run the command, the command will ask your email and password and later you will see something like that in your screen:
Downloading data records.
[INFO ] Logging to bulkloader-log-20110227.162029
[INFO ] Throttling transfers:
[INFO ] Bandwidth: 250000 bytes/second
[INFO ] HTTP connections: 8/second
[INFO ] Entities inserted/fetched/modified: 20/second
[INFO ] Batch Size: 10
[INFO ] Opening database: bulkloader-progress-20110227.162029.sql3
[INFO ] Opening database: bulkloader-results-20110227.162029.sql3
[INFO ] Connecting to hsvoxpro.appspot.com/remote_api
Please enter login credentials for [YOUR APP]
Email: [YOUR EMAIL]
Password for [YOUR PASSWORD]
[INFO ] Downloading kinds: [u'Category', u'UserData', u'MailQueue', u'Group', u'Item', u'UserSubscription', u'Task',
u'Application', u'Follower', u'Tag', u'GroupItem', u'Vote', u'GroupUser', u'Event', u'ItemHtml']
.............[INFO ] Category: No descending index on __key__, performing serial download
[INFO ] UserSubscription: No descending index on __key__, performing serial download
[INFO ] Item: No descending index on __key__, performing serial download
[INFO ] Task: No descending index on __key__, performing serial download
[INFO ] Follower: No descending index on __key__, performing serial download
[INFO ] Tag: No descending index on __key__, performing serial download
...............
[INFO ] Have 195 entities, 0 previously transferred
[INFO ] 195 entities (400322 bytes) transferred in 24.0 seconds
Import the exported data to a datastore is easy too. The follow example shows you how to import the data in your SDK datastore:
[APP_ENGINE_SDK_FOLDER]>appcfg.py upload_data --filename=data.exp --url=http://localhost:8080/remote_api [APP_LOCAL_FOLDER]
Where [APP_ENGINE_SDK_FOLDER] is your Google's Appengine SDK installation folder and [APP_LOCAL_FOLDER] the folder that contains your Vikuit's installation.
If you run the command, you will see something like that in your screen:
Application: [YOUR APP]; version: 2.
Uploading data records.
[INFO ] Logging to bulkloader-log-20110227.163459
[INFO ] Throttling transfers:
[INFO ] Bandwidth: 250000 bytes/second
[INFO ] HTTP connections: 8/second
[INFO ] Entities inserted/fetched/modified: 20/second
[INFO ] Batch Size: 10
[INFO ] Opening database: bulkloader-progress-20110227.163459.sql3
Please enter login credentials for localhost
Email: [YOUR EMAIL]
[YOUR PASSWORD]
[INFO ] Connecting to localhost:8080/remote_api
[INFO ] Starting import; maximum 10 entities per post
....................
[INFO ] 195 entites total, 0 previously transferred
[INFO ] 195 entities (358347 bytes) transferred in 45.3 seconds
[INFO ] All entities successfully transferred
The bulk loader can be tuned easily.For a more detailed information about it, please check it the Google's page:
