Recently I started a new project. There are a lot of variables that change based on client needs. We are ignoring a lot of those aspects. We are cutting right to massaging the data into the appropriate formats. Pipeline stationing often has a plus in it, which is frustrating because Arcmap won’t directly read this column when creating a layer for this data. Because when engineers measure in the field they measure with chains in 100 feet increments. So the data often has a plus in it to show what section of chain they are using.
If we are working with a single table there’s a find and replace function that we can utilize. Its located in options in Arcmap 9.3
Find and replace located under options in a table.
If we are working with a lot of tables, it might not make sense to open each table and run the find and replace on each table. Model builder is probably the easiest way to overcome this problem. Since we are modifying the original data, the best practice to to create a new column,
Adding a field to a table.
and then field calculate with the replace function. VB code goes in the expression box. If you need help writing the expression you can google for VB and what you are trying to do with it.
This is the VBA code
I pulled a screenshot from google to show how the vb code looks before we modify it to fit our expression needs. String1 in this case is the column we will be looking into to run our replace expression, the + is what we are looking to replace. Notice that the replacement is left null. I have ignored the rest of the replace statement because it is not relevant to our calculate.
If you search for a find and replace in the tools, you won’t find it. Using the find and replace function actually takes writing some VisualBasic code inside of our field calculation.
Model builder showing the process of modifying the testpoint table.
If you have never built a model before I suggest doing a walkthrough on the ESRI website.
When you are delivering to a client a GIS, these are typically the items being delivered.
- logical model – This is a ‘roadmap’ for your entire GIS. It will show the table relationships and what tables are subtables of tables. It helps you keep your database logical and working.
- physical model design – uml, or database. This is what people typically assume is the GIS. It is what is displayed in Arcmap.
- change log – This tracks not only what is happening in your database but what is happening to your logical and physical model. Chances are you didn’t map your data correctly and have had to move some relationships around. This helps you track what you moved, and most importantly why.
data dictionary – this describes the tables. Thing such as attributes, domains, relationships fall into this portion of the data model.
This is a quick technical post. There are 3 big tips when coping out a personal or file geodatabase.
1) The quickest, and most painless way to copy one geodatabase to another is to open in ArcCatalog, and drag and drop the files.
2) Remove relationships from the data set. We use relationships to be able to link tables together. They have a special designation in ArcMap and will stop your data from copying correctly. Unselect them when you go to drag and drop them into your new geodatabase.
3) Don’t copy/paste any views, unselect these as well.
The biggest take away from this post is to not blindly select all and try to copy into another database. Be specific about what you grab.
To be completely honest, this post is a very high level. You need to know data models exist and which one has been implemented on your project. Pro’s Con’s of each don’t really matter because youre stuck with what your project began with and what the client wanted. BUT if you ever hope to someday be the one picking the data models to use for your projects, then you better know which one fits your project.
Why is data modeling important to you.
There are three industry standard data models. It is a template for you to follow when building a GIS. None of the three data models are perfect. It basically dictates what table your data goes into, and how it relates to other tables.
What is a data model?
At its basic level it helps you keep track of where you are storing things in your database. It is a visual representation of your database relationships.
Knowing your data models.
This post is going to teach you the very basic of the three different data models currently in use within the Oil and Gas Industry.
What are three data models?
I’m already confused, what’s the difference between the first two?
I will give each different type of data model its own post.
When people talk about GIS and the pipeline industry they are really talking about
pipeline integrity management
oooooooh, industry buzz words! Let’s talk candidly about what integrity management means. GIS is only a small portion of what constitutes pipeline integrity management. The pipeline industry has made big changes from past practices. Government regulation has *helped* boost the GIS industry to new heights. The regulations came about because of pipeline accidents. From the US Government Transportation Pipeline and Hazardous Safety Website
The integrity management rule has four primary objectives:
- accelerating the integrity assessment of pipelines in High Consequence Areas
- improving operator integrity management systems
- improving government’s role in reviewing the adequacy of integrity programs and plans, and
- providing increased public assurance in pipeline safety.
The government decided that pipeline operators needed to validate what they operate, and what is in the ground. Rather than dig up every single pipeline, operators needed a way to show that they met the federal regulations. In comes GIS to save the day! If you strip GIS down to its basic core, removing all the fancy symbology, and layers, what are you left with? A database, but it’s MORE than just a database. It’s a spatial database, meaning the database is tracking the events in relation to where they are in the world. Pipeline operators can track the location of items in their database (think pipeline). They are able to track updates to their pipeline in the database. Most importantly, operators are able to link engineering documents or other important documents to the actual pipelines that they relate to. These rules are the reason for a GIS, how the data is managed inside of that database is often left up to the pipeline owner. Which is how we end up with different data models.
My hope for this website is to be the blog I had wished for when I started my career mapping pipelines. GIS as an industry has a lot of breadth and depth, more than any one person can cover. There are a lot of GIS industry specific things that I won’t be tackling. For instance, I don’t do Spatial analysis on a daily basis, so I won’t be focusing on that. I do digitize a lot of pipeline features in Arcmap and load a lot of data. There are a lot of Pipeline industry specific things I won’t be covering as well.
I know it isn’t sexy or exciting, but I have learned a lot about the Oil and Gas industry. So my GIS blog will be somewhat industry specific. I will try to branch out as I continue to advance my career.