Data Migration Project Life Cycle

295 pts.
Tags:
Data migration
Life cycle management
QA
Software testing
What is Data Migration Project Life Cycle?

Answer Wiki

Thanks. We'll let you know when a new response is added.

Two questions in one here:

Project lifecycle and data migration; data migration is moving data form one system into another system, but more on this later below. Project Lifecycle is a process of steps taking a set of interrelated tasks form one point to another.

A project lifecycle covers (but not limited to) the following:
<ol>
Project inception
Production of a business case
Production of a design
Definition of the stages and related tasks
Selection of the team or vendor
Stage one (the stages and the tasks will be covered in the data migration part of this below)
Tasks one through to x
Stage two
Tasks from x to z
Sign off of the project delivered what was in the product design
Project closure (this looking at what went right and wrong and how the project did and using this as learning for the next project)
[?OLIST]

I’ve had project plans that are not much bigger than this to project plans with 4000 lines and a project can take a week or years, some construction projects can take decades, in IT the enhancements in technology and project that is going to take more than 18 months will need to have technology and business reviews to make sure it is still delivering the requirements and the business model or goals have not moved on. I was asked to run a project to install and set up an Oracle db but with the time scales (due to the clients budget and resourcing issues) the installed version of oracle they wanted would only have a 18 month support period before oracle started charging more for support or expect clients to upgrade to the next version. This project did not happen,

Data migration, can be simple and poor adding no value just using up money and resource or can be easy and adds value to the business as a whole. The one main thing to remember is the long you take between cleaning of the data and the actual migration there will be a big move in data between the cleaned and the existing systems, from cleaned and shortfall data fulfilment to the migration and users in the system should be no longer than 5days at worst and if possible do it al in the same migration process, I have asked for a system from Friday afternoon (12 o’clock) to Monday morning (9am) and had a team working all the time including through the night to do it all on a large db or multiple dbs/systems into one system

There are simple rules to data migration:
<ol>
back up the data
test the restore of the back up
understand the data
understand the data source
understand the data designation
identify shortfall of your data (this could be as simple as most of the records don’t have an email address vital for a CRM solution, or the postcodes (zip codes) are not complete, you have no first names only surnames, or a credit rating for customers. There are many companies which can supply this data, phone numbers cost about a penny per number checked and 10 pence per shortfall, to email address at 50 pence or more per email address, shop around and never pay for shortfall checking, the companies should do this for free, I have never paid, but know some of the big consultancies do charge, but they have to pay for the big offices some how)
back up the data
clean the data (both manually and automatically, I have never seen an automatic clean do all the data, it is some times cheaper to manually clean some data than it is to write scripts that are hundreds of lines long when a human could do it in less time than it takes to write the script and test it fully)
back up the data
de-dup the data (you can either clean or du-dup and some people cal it the same thing
back up the data
add short fall data
developed the migration scripts (it may take two or more moves to get the data into the right format for the new system, it is better to take as many steps to do this as necessary, because the data will be in the right place and the data will be usable for the end user)
test a small sub set of the data into the new system (this one catches so many projects out they move it all over then test and find they have major problems which normal ends up with them having to reinstall and start all over again)
run a number of trail runs and time them and list down the steps on a sheet (this allows you to make sure the migration can be done in the system down time allowed, I did one at a bank which had to be done between Friday nigh 11 o’clock and had to be working by Saturday 6 pm, if not it all had to be restored for 1 pm on Sunday for the first of the automated transfers on Sunday night. A sheet also helps in you list down how long each bit takes and what happens, some times you will get a record which causes an error, but you have to allow the error because of the data, so if this happens you know the process is working if it does not then you know some thing has gone wrong and if the timings are way out this is a good pointer to problems)
develop a regression script to check data
test the regression script
run the migration
[/OLIST

If al has gone well then sit back and enjoy the praise and say if this happens again I’m off, until it comes around again and you then dig out the check sheets and what went wrong and start planning all over again.

The above answer takes care of most of the technical stuff (?) but leaves out a very vital part in the data migration which is reconciliation. there should be a process to ensure that what has been moved in from one system to the other (be it a simple database version upgrade or a total change of system / package) is the same. For eg., while moving a customer data, the number of customers grouped by sex in the source system and the target system need to be the same. It is more simple telling than doing. For eg., the sex may be represented by ‘M’ and ‘F’ (ignoring all other classifications like Corporate, etc) in the source system but might be 1 and 2 in the target system. So the mapping of data values between the two systems need to be done prior to migration. And it should be ensured that it was done correctly through migration. Which is what reconciliation is all about. And when it comes to reconciling financial data it should be ensured that there is 100% accuracy. Many trial runs need to be done and the reconciliation should be conducted. Reconciliation, by itself, is a separate elaborate process which needs proper understanding of the source and target systems as well as the migration approach.</ol></ol>

Discuss This Question: 1  Reply

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when members answer or reply to this question.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • MEHRA
    Hi , can any tell me Which tool we are using for data migration? Is there is any external tool for data migration or simply run the script to different machine?
    145 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

To follow this tag...

There was an error processing your information. Please try again later.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Thanks! We'll email you when relevant content is added and updated.

Following