We all know how useful data migration is for project management, making it easier for stakeholders to work together and improving decision-making processes.
But in the fast-changing world of architecture, engineering and construction (AEC), data migration is much more than moving numbers and files. It’s the foundation of digital transformation.
That’s why Australian AEC firms have to deal with a few particular issues regarding data migration.
One of the main challenges is dealing with data that’s spread across different systems and formats, which can make the migration process more complicated.
On top of that, the costs involved in data migration and the need for specific skills can be a real barrier for some firms, especially small and medium-sized businesses (SMEs).
Security is also a big issue, with things like making sure data is safe and secure during migration, and protecting against breaches.
But there are also some great opportunities. Right, let’s run through the lowdown on data migration in Australia’s AEC industry.
What is Data Migration?
Data migration is a fancy but simple way of saying moving data from one system to another.
This could mean moving data from old legacy systems to modern cloud-based platforms, combining data from different sources into one place, or even transferring data between different software applications used on a project.
So, there are different types of data migration, each with its own particular challenges.
Cloud-to-cloud migration is about moving data between different cloud platforms. Meanwhile, on-premise to cloud migration is all about shifting data from local servers to a cloud environment.
There are other types, too, such as application migration, storage migration, and database migration.
Therefore, you should know the type of migration you choose will depend on what your AEC firm needs and what’s going on in a particular situation.
The Evolving Landscape of Data Migration in Australian AEC (2009-2024)
The journey of data migration in the Australian AEC industry over the past 15 years or so has been one of gradual change, with a few big jumps along the way.
In the early years, from 2009 to around 2015, data migration was often seen as a bit of a hassle. It was mainly done when upgrading software or hardware.
The main goal was to make sure the data was safe and to cut down on time when things were down, with not much thought given to using the data to gain a strategic advantage.
However, as Christian Criado-Perez et al., highlight in their 2022 paper “Digital Transformation in the Australian AEC Industry: Prevailing Issues and Prospective Leadership Thinking,” the landscape began to change around 2016-2017.
The rise of Building Information Modelling (BIM) and other digital technologies showed how valuable data can be, not just as a record, but as a strategic asset too.
Data migration wasn’t just about keeping things safe; it was about seeing what else we could do with it.
This shift was further catalysed by the growing awareness of Industry 4.0, as explored by Sahar Soltani et al., in their 2023 study “The State of Industry 4.0 in the Australian Construction Industry: An Examination of Industry and Academic Point of View.”
The integration of digital and physical systems meant we needed to make sure data flows were smooth, so data migration was a key part of getting Industry 4.0 up and running.
From 2018 onwards, data migration in the Australian AEC industry has become much more strategic and sophisticated.
Firms are putting money into solid data migration plans, seeing that getting it right isn’t just about technical know-how.
It’s also about managing change, engaging with stakeholders, and having a clear idea of how the migrated data will be used to drive innovation and growth.
The Engineers Australia Annual Report for 2021-2022 shows how the organisation is investing a lot in modernising its data and digital capabilities.
It shows that there’s been a 10.33% increase in total membership and a 9.31% increase in National Engineering Register (NER) registrants.
That number means there’s a growing demand for digital services and data-driven insights.
Looking ahead to 2024, it’s pretty clear where things are headed. Data migration is going to be a key part of the digital transformation of the Australian AEC industry.
If you can master data migration, you’ll be in a great position to make the most of data, drive innovation and succeed in the digital age.
But to make it work, you’ll need to think about more than just the technical side of things.
You’ll also need to consider things like strategic vision, change management and a commitment to learning and adapting.
Anyway, you may want to read the “AEC Data Migration Rules” to better understand the data migration conditions in Australia.
Why Migrate Data?
Looking at how data migration has evolved over the past 15 years, we’re clearly talking more than just a simple IT housekeeping task here.
The research by Engineers Australia, as detailed in their 2021-2022 Annual Report, shows how making data accessible can transform things.
They invested in modernising their data, leading to a big increase in membership and engagement. This shows how breaking down data silos and encouraging collaboration can help a business to grow.
For AEC firms, this means easier workflows, with project teams able to access and update important information as it happens, no matter where they are or what they do.
Another great reason to migrate is to boost efficiency and productivity.
Soltani et al., highlight the importance of data-driven construction and automation in achieving Industry 4.0 goals.
By moving to new, modern systems, AEC firms can make the most of their data.
They can use analytics to spot bottlenecks, maximise their resources, and automate some of the more routine tasks.
Data migration also has big financial benefits. Legacy systems can be a real drain on resources, especially when they have high maintenance costs and limited scalability.
The shift to cloud-based solutions, as advocated by Criado-Perez et al., offers a compelling alternative.
Firms can save a lot of money by getting rid of the need for on-site hardware and making their IT infrastructure more efficient.
And finally, data migration isn’t just about seizing opportunities; it’s also about managing risks.
Moving to new systems with strong security and encryption protects sensitive project data, which helps firms avoid the financial and reputational damage of data breaches.
Don’t worry and hesitate. There are many successful examples as you can see in this “Data Migration Case Study“.
Preparing for Data Migration
Data Assessment and Inventory
Before moving, you wouldn’t pack up everything in sight without first sorting through what you really need and what you can leave behind.
Similarly, a thorough data assessment is all about identifying and cataloguing all the data that’s relevant to your operations.
This means taking a look at all the data assets you have, including databases, files, applications, and storage systems.
The inventory should include details of the data types, volumes, formats and dependencies.
Knowing what data you’ve got, where it’s from and how it’s used is really valuable. It means you can make sure you don’t miss anything important when you’re moving it over.
Data cleansing and deduplication are integral steps in this phase.
Data cleansing is about fixing or getting rid of inaccurate, corrupted, or incomplete data.
Deduplication is a way of combining duplicate copies of data to make better use of storage space and improve data quality.
Defining Migration Objectives
We need to figure out which data we’re going to migrate, set a timeline, and decide on a budget.
This means setting goals to be specific, measurable, achievable, relevant and time-bound (SMART).
For instance, you might decide to move all the project data from the last five years, get it all done in six months, and stick to a budget you’ve already set.
A good plan will cover everything you need to know about the migration, including which data you’re moving and which systems it’s going to affect.
It’s also a good idea to think about how this could affect the business and come up with ways to avoid any hiccups.
So, it’s also important to make sure all stakeholders know where they stand and what they need to do.
Choosing a Migration Strategy
There’s no one-size-fits-all approach to data migration. That’s why there are a few things to think about when you’re choosing a strategy.
Things like how big and complex your data is, what resources you have available, and how much risk you can handle. So, we’ve put together a few of the most popular migration strategies.
Big Bang Migration
This approach means you can get the whole migration done in one go, which is pretty quick.
It’s usually quicker and cheaper, but there are more risks involved because there’s a chance the system will go down and there might be data transfer errors.
This method is a good fit when you can schedule downtime and keep a tight rein on the migration.
Phased Migration
You might also know this as ‘trickle migration’ – it’s a strategy that involves moving data in stages.
It lets you run systems in parallel during the migration process, which helps to avoid any disruption.
This approach is a bit more complex and time-consuming, but it offers more flexibility and helps to keep business operations running smoothly.
This method lets you test and validate at each stage, which can really help you avoid errors.
Hybrid Migration
Hybrid migration is a great way to balance speed and risk. It combines elements of both big bang and phased approaches.
For instance, you could move the core data in one go, while less critical data is migrated in phases.
You can adapt this strategy to suit your company’s particular needs and limitations, creating a flexible and reliable migration plan.
Data Migration Process: Step-by-Step
Data migration is a pretty complex but crucial process for Australian AEC. The process below is a basic overview of what you’ll be doing when you take on data migration in your business.
Data Extraction
Once the groundwork is laid, the next step is data extraction.
This involves retrieving data from the source systems, which could be anything from outdated legacy software to on-premise file servers.
The extraction process needs to be carefully managed to ensure data integrity and minimise disruption to ongoing operations.
Depending on the source system, this might involve using specialised extraction tools or writing custom scripts to access and retrieve the data.
Data Transformation
Once extracted, data needs to be transformed into a format compatible with the target system.
This might involve changing data schemas, reformatting data types, or converting data structures to match those of the new environment.
For example, when migrating to a BIM platform like Autodesk’s BIM 360 or Autodesk Construction Cloud (ACC), the data might need to be transformed to align with Autodesk’s specific data schemas.
Data Loading
With the data transformed, it’s time for data loading. This involves importing the transformed data into the target platform.
The loading process needs to be efficient and reliable, ensuring that all data is transferred accurately and completely.
Depending on the volume of data and the target system’s capabilities, this might involve using bulk loading tools or incremental loading techniques.
Data Validation and Verification
Post-migration, it’s crucial to validate and verify the data to ensure accuracy and completeness.
This involves running tests to check for data integrity, consistency, and completeness.
User Acceptance Testing (UAT) is a key component, involving end-users to confirm that the data meets business requirements and functions correctly in the new system.
Post-Migration Optimization
Data migration is not the end; it’s a new beginning.
The post-migration phase involves optimising the new environment to ensure that the migrated data is being used effectively.
This might involve fine-tuning system configurations, developing new workflows, and providing user training to ensure smooth adoption.
Ongoing data management is also crucial to maintain data quality, ensure data security, and adapt to evolving business needs.
For a complete guide, please read “An Australia Step-by-Step Guide to Autodesk Construction Cloud Migration.”
Best Practices of Data Migration in 2024
The golden rule is to make sure the data is in good shape before you start migrating it.
Absolutely. Collecting and ensuring the quality of data during migration is definitely a challenge.
Migrating inaccurate or incomplete data can end up costing you more in the long run.
So, it’s crucial to have a solid data cleansing and normalisation process.
This means spotting and fixing any inconsistencies, errors and redundancies in the data before migration.
It’s about making sure the data you’re moving is not just there in plenty, but also clean and ready to use.
Tools like Autodesk Info360 Insight can help you profile and clean up your data.
Then, make sure you don’t let data migration bring your operations to a standstill.
The aim is to keep things running smoothly and make sure the switch to the new system is as seamless as possible.
This means planning carefully, like scheduling the migration during off-peak hours or weekends, and having a solid rollback plan in case something unexpected comes up.
That’s why it’s vital to focus on change management and communication during digital projects.
Keeping your stakeholders informed and involved throughout the migration process can help manage expectations and ensure a smooth transition.
Keeping your stakeholders on side with effective stakeholder management is the key to getting them to buy in and support you.
This means keeping everyone in the loop with clear communication, regular updates and addressing any concerns or resistance to change.
Regulatory and Compliance of Data Migration in Australia
The Australian regulatory landscape isn’t as strict as in some other places, but it’s still pretty complex for AEC firms to navigate when they’re moving data.
There isn’t one unified national framework for data privacy and security, which can create some ambiguity and challenges.
Fortunately, there are a few key regulations and standards that provide guidance.
The Privacy Act 1988 is all about how we handle personal information, while the Security of Critical Infrastructure Act 2018 is about protecting critical infrastructure from cyber threats.
The 2023 Privacy Act Review Report has put forward 116 changes, 38 of which have been fully accepted by the government and 68 of which have been agreed in principle.
These changes are the biggest update since the Australian Privacy Principles (APPs) were introduced.
The new laws will mean businesses have to set out how long they keep data, make it easier for people to give consent, and be more open about automated decision-making.
So, companies will need to get more serious about data protection and provide clearer, more concise privacy notices.
On the other hand, the Office of the Australian Information Commissioner (OAIC) will have more power to enforce the rules, including the ability to impose big penalties for not following them.
Tools and Technologies as the Engines of Data Migration in AEC
Migrating data in the AEC industry is a complex process, but we’ve got a good set of tools to help us along the way.
There are loads of data migration tools out there, from the tried and tested enterprise solutions to the more agile open-source options.
Each one has its own strengths and specialisms. Some of the most popular choices in the AEC space include:
- Autodesk BIM 360 and ACC: These cloud-based platforms are designed specifically for AEC, offering solid data management and collaboration features.
- Oracle Data Integrator: This data integration platform is great for handling complex migration scenarios. It has features like data profiling, cleansing, and transformation.
- Informatica PowerCenter: Another top-notch enterprise solution, PowerCenter, offers a visual interface for designing and executing data migration workflows.
- Pentaho Data Integration: Its simple drag-and-drop interface and huge library of plugins make it easy to use, even if you’re not a tech expert.
Detailed Look at BIM360 and ACC
We’ll be upfront with you – we’re Autodesk Gold Partners at Interscale, so our perspective might be slightly biassed.
But as Autodesk Gold Partners, we’ve first-hand experience with these platforms, and we’re keen to share what we’ve learned.
BIM 360 and Autodesk Construction Cloud (ACC) are designed specifically for the AEC industry, and they’re the perfect central hub for all your project data.
They’re the ultimate organiser for your Revit models, CAD drawings, spreadsheets and more – everything in one place, accessible from anywhere.
Yes, these tools are not your average cloud storage solutions.
One of the best features is the 360 Admin Tools. These handy little tools take care of those annoying admin tasks we all hate, like user assignments and project syncing.
This means you can focus on the important stuff – like actually designing and building things.
But it’s the integration capabilities of BIM 360 and ACC that really make them stand out.
These platforms work well with lots of AEC software, including the big names like AutoCAD, Revit, and Civil 3D.
This means you can easily move data between these apps, making collaboration a breeze.
Let’s look at the Autodesk AutoCAD Connector as an example. This nifty little tool makes it super easy to exchange data between AutoCAD and other Autodesk software.
So, you don’t have to email files back and forth or worry about version control – it’s all handled in the cloud.
Challenges of Data Migration in Australia AEC Industries
The amount of data AEC firms generate and store over time can be pretty overwhelming.
Old systems, which are often fragmented and siloed, make it even more difficult to extract and transform the data.
The tricky part is not just moving the data, but also making sure it’s accurate, complete and compatible with the new system.
The answer is usually a mix of technical know-how and strategic planning.
Tools for cleaning up and normalising data can help you spot and fix inconsistencies, while careful data mapping makes sure everything integrates seamlessly with the target system.
On top of that, using Extract, Transform, and Load (ETL) techniques can help you manage large datasets by segmenting and prioritising the data during migration.
Then there are the security and compliance issues. The Australian regulatory landscape is changing, but it still requires very strict data protection measures.
Just a couple of examples of the regulations that AEC firms have to stick to are the Privacy Act 1988 and the Security of Critical Infrastructure Act 2018.
The answer is to make security your top priority throughout the migration process.
Encryption, access controls and secure data transfer protocols are non-negotiables.
It’s also a good idea to team up with data migration experts who know the ins and outs of the Australian regulatory landscape. They can give you some great advice and help you stay compliant.
Future Trends of Data Migration in Australia AEC Industry
AI, Cloud, and Automation
Artificial intelligence (AI), cloud computing and automation are transforming industries in big ways.
AI-powered tools can automate data profiling, cleansing and transformation, which makes migration a lot quicker and easier.
Cloud-based migration solutions are scalable, flexible and cost-effective, so firms can handle large data volumes without having to invest in on-premise infrastructure.
Automation can make repetitive tasks easier and more accurate, which helps to keep data safe and sound.
BIM Influence
Building Information Modelling (BIM) is still having a big impact on the Architecture, Engineering, and Construction (AEC) industry.
As more and more companies start using BIM, data migration will become more and more linked to BIM workflows.
BIM is being used more and more with AI and cloud tech, which is making project management and design processes better.
For example, AI-driven BIM platforms can predict potential delays and cost overruns by looking at historical data, which helps to improve project outcomes.
So yes, the future looks bright for data migration in AEC, with lots of exciting developments on the way. We can expect to see:
- Increased adoption of cloud-based migration solutions.
- Greater reliance on AI and automation.
- Seamless integration with BIM workflows.
- Enhanced focus on data security and compliance.
In Closing
Over the past 15 years, data migration has gone from being a routine process to being a key part of how we innovate and work more efficiently.
But, as with anything, there are a few hurdles to overcome along the way.
Migrating large amounts of data, making sure we comply with the rules and keeping our systems secure all require careful planning and execution.
While there are still a few hurdles to overcome, the future of data migration in the Australian AEC industry looks bright. We’re on the cusp of a new era of efficiency, productivity and data-driven decision-making.
