Renaming columns using geopandas - issues with data table in ArcGeoPandas to_file() saves GeoDataFrame without coordinate systemFrom text data create a FeatureClass with table with pythonSpeed up row-wise point in polygon with GeopandasRenaming shapefiles using loop in PythonGeopandas: Write layer back into GeoDataBaseConverting shapefile data to .osm using ogr2osmGeopandas not recognizing geometry typeHow do I write a GeoPandas dataframe into a single file (preferably JSON or GeoPackage)?gdal/geopandas data object compatibility in pythonSorting (ORDER BY) rows from Shapefile using Python?How to create a shapefile [polygon type] from a Geodataframe, returned from a Oracle Spatial cursor with geometry column type=cx_Oracle.LOB?

Go Pregnant or Go Home

System.debug(JSON.Serialize(o)) Not longer shows full string

Was the picture area of a CRT a parallelogram (instead of a true rectangle)?

Lay out the Carpet

Is there a good way to store credentials outside of a password manager?

Gears on left are inverse to gears on right?

Is there a problem with hiding "forgot password" until it's needed?

How do I rename a Linux host without needing to reboot for the rename to take effect?

Can criminal fraud exist without damages?

What to do with wrong results in talks?

How to safely derail a train during transit?

What is the best translation for "slot" in the context of multiplayer video games?

What can we do to stop prior company from asking us questions?

How does Loki do this?

Unexpected indention in bibliography items (beamer)

How did Doctor Strange see the winning outcome in Avengers: Infinity War?

What's this thing which looks like a water sensor inside a PC mouse?

What is the difference between "behavior" and "behaviour"?

Sort a list by elements of another list

You cannot touch me, but I can touch you, who am I?

What is the opposite of 'gravitas'?

How did Arya survive the stabbing?

How to draw lines on a tikz-cd diagram

How can I kill an app using Terminal?



Renaming columns using geopandas - issues with data table in Arc


GeoPandas to_file() saves GeoDataFrame without coordinate systemFrom text data create a FeatureClass with table with pythonSpeed up row-wise point in polygon with GeopandasRenaming shapefiles using loop in PythonGeopandas: Write layer back into GeoDataBaseConverting shapefile data to .osm using ogr2osmGeopandas not recognizing geometry typeHow do I write a GeoPandas dataframe into a single file (preferably JSON or GeoPackage)?gdal/geopandas data object compatibility in pythonSorting (ORDER BY) rows from Shapefile using Python?How to create a shapefile [polygon type] from a Geodataframe, returned from a Oracle Spatial cursor with geometry column type=cx_Oracle.LOB?













1















What I'm trying to achieve:



  • Read in an existing ESRI polygon shapefile

  • Drop an attribute

  • Dissolve by another attribute

  • Rename columns

  • Reorder columns

  • Write out as new shapefile

What works: Drop column and dissolve all work - I've run these steps and written to file. This opens fine in ArcMap, I can identify things, open the attribute table etc. No issues.



Problem: When I rename (not got to the reorder step without error yet) the columns, everything runs fine and I write to file. When then opening the shapefile in ArcMap (10.3.1) I can query layers using the Identify cursor which shows my renamed attributes as they should be BUT when trying to open the attribute table, I get the following message in a window:



Loading table data



Could not load data from the data source. If you can correct the problem, press the refresh button to reload data. Possible problems can include bad network connection, invalid file, etc.



A column was specified that doesn't exists.



A column was specified that doesn't exists.



So, it looks like something is missing or my changes are not being mapped across the shapefile components. I can't share the data but can share the code. The problem is with the renaming... What am I missing?



import geopandas as gp

# Read data in
f="my_file"
data = gp.read_file(f)

# Drop a column (if writing out file here, does as expected)
data=data.drop(['CLASS'], axis=1)

# Do the dissolve (if writing out file here, does as expected)
data=data.dissolve(by='Hazard_Pot', as_index=False)

# Rename (THE PROBLEM I THINK : if saving here, I get my issue in ArcMap)
data=data.rename(index=str, columns="Hazard_Pot":"Potential",
"casualties":"Casualty",
"vehicle_ac":"Access",
"building d":"Damage",
"disruption":"Disruption",
"contaminat":"Contam.",
"do_not_1":"Do_Not_1")

# Reorder
data=data[['Potential',
'Casualty',
'Access',
'Damage',
'Disruption',
'Contam.',
'Do_Not_1',
'geometry']]

# write to file (creating a new CRS based on the old as per https://gis.stackexchange.com/questions/204201/geopandas-to-file-saves-geodataframe-without-coordinate-system?noredirect=1&lq=1)
prj_file = "%s.prj" %os.path.splitext(f)[0]
prj = [l.strip() for l in open(prj_file,'r')][0]
data.to_file(ofile, driver=driver, crs_wkt=prj)









share|improve this question






















  • data.rename(index=str... renaming the index to a data type is very strange. Take that part out and report back

    – Paul H
    4 hours ago











  • I can't share the data but can share the code and you can share fake data that reproduces the issue

    – Paul H
    4 hours ago











  • @PaulH i belive index=str is correct syntax, see the examples in help section of rename. But I also think that part is not needed.

    – BERA
    4 hours ago












  • @BERA Oops - forgot rename could take callables.

    – Paul H
    3 hours ago















1















What I'm trying to achieve:



  • Read in an existing ESRI polygon shapefile

  • Drop an attribute

  • Dissolve by another attribute

  • Rename columns

  • Reorder columns

  • Write out as new shapefile

What works: Drop column and dissolve all work - I've run these steps and written to file. This opens fine in ArcMap, I can identify things, open the attribute table etc. No issues.



Problem: When I rename (not got to the reorder step without error yet) the columns, everything runs fine and I write to file. When then opening the shapefile in ArcMap (10.3.1) I can query layers using the Identify cursor which shows my renamed attributes as they should be BUT when trying to open the attribute table, I get the following message in a window:



Loading table data



Could not load data from the data source. If you can correct the problem, press the refresh button to reload data. Possible problems can include bad network connection, invalid file, etc.



A column was specified that doesn't exists.



A column was specified that doesn't exists.



So, it looks like something is missing or my changes are not being mapped across the shapefile components. I can't share the data but can share the code. The problem is with the renaming... What am I missing?



import geopandas as gp

# Read data in
f="my_file"
data = gp.read_file(f)

# Drop a column (if writing out file here, does as expected)
data=data.drop(['CLASS'], axis=1)

# Do the dissolve (if writing out file here, does as expected)
data=data.dissolve(by='Hazard_Pot', as_index=False)

# Rename (THE PROBLEM I THINK : if saving here, I get my issue in ArcMap)
data=data.rename(index=str, columns="Hazard_Pot":"Potential",
"casualties":"Casualty",
"vehicle_ac":"Access",
"building d":"Damage",
"disruption":"Disruption",
"contaminat":"Contam.",
"do_not_1":"Do_Not_1")

# Reorder
data=data[['Potential',
'Casualty',
'Access',
'Damage',
'Disruption',
'Contam.',
'Do_Not_1',
'geometry']]

# write to file (creating a new CRS based on the old as per https://gis.stackexchange.com/questions/204201/geopandas-to-file-saves-geodataframe-without-coordinate-system?noredirect=1&lq=1)
prj_file = "%s.prj" %os.path.splitext(f)[0]
prj = [l.strip() for l in open(prj_file,'r')][0]
data.to_file(ofile, driver=driver, crs_wkt=prj)









share|improve this question






















  • data.rename(index=str... renaming the index to a data type is very strange. Take that part out and report back

    – Paul H
    4 hours ago











  • I can't share the data but can share the code and you can share fake data that reproduces the issue

    – Paul H
    4 hours ago











  • @PaulH i belive index=str is correct syntax, see the examples in help section of rename. But I also think that part is not needed.

    – BERA
    4 hours ago












  • @BERA Oops - forgot rename could take callables.

    – Paul H
    3 hours ago













1












1








1








What I'm trying to achieve:



  • Read in an existing ESRI polygon shapefile

  • Drop an attribute

  • Dissolve by another attribute

  • Rename columns

  • Reorder columns

  • Write out as new shapefile

What works: Drop column and dissolve all work - I've run these steps and written to file. This opens fine in ArcMap, I can identify things, open the attribute table etc. No issues.



Problem: When I rename (not got to the reorder step without error yet) the columns, everything runs fine and I write to file. When then opening the shapefile in ArcMap (10.3.1) I can query layers using the Identify cursor which shows my renamed attributes as they should be BUT when trying to open the attribute table, I get the following message in a window:



Loading table data



Could not load data from the data source. If you can correct the problem, press the refresh button to reload data. Possible problems can include bad network connection, invalid file, etc.



A column was specified that doesn't exists.



A column was specified that doesn't exists.



So, it looks like something is missing or my changes are not being mapped across the shapefile components. I can't share the data but can share the code. The problem is with the renaming... What am I missing?



import geopandas as gp

# Read data in
f="my_file"
data = gp.read_file(f)

# Drop a column (if writing out file here, does as expected)
data=data.drop(['CLASS'], axis=1)

# Do the dissolve (if writing out file here, does as expected)
data=data.dissolve(by='Hazard_Pot', as_index=False)

# Rename (THE PROBLEM I THINK : if saving here, I get my issue in ArcMap)
data=data.rename(index=str, columns="Hazard_Pot":"Potential",
"casualties":"Casualty",
"vehicle_ac":"Access",
"building d":"Damage",
"disruption":"Disruption",
"contaminat":"Contam.",
"do_not_1":"Do_Not_1")

# Reorder
data=data[['Potential',
'Casualty',
'Access',
'Damage',
'Disruption',
'Contam.',
'Do_Not_1',
'geometry']]

# write to file (creating a new CRS based on the old as per https://gis.stackexchange.com/questions/204201/geopandas-to-file-saves-geodataframe-without-coordinate-system?noredirect=1&lq=1)
prj_file = "%s.prj" %os.path.splitext(f)[0]
prj = [l.strip() for l in open(prj_file,'r')][0]
data.to_file(ofile, driver=driver, crs_wkt=prj)









share|improve this question














What I'm trying to achieve:



  • Read in an existing ESRI polygon shapefile

  • Drop an attribute

  • Dissolve by another attribute

  • Rename columns

  • Reorder columns

  • Write out as new shapefile

What works: Drop column and dissolve all work - I've run these steps and written to file. This opens fine in ArcMap, I can identify things, open the attribute table etc. No issues.



Problem: When I rename (not got to the reorder step without error yet) the columns, everything runs fine and I write to file. When then opening the shapefile in ArcMap (10.3.1) I can query layers using the Identify cursor which shows my renamed attributes as they should be BUT when trying to open the attribute table, I get the following message in a window:



Loading table data



Could not load data from the data source. If you can correct the problem, press the refresh button to reload data. Possible problems can include bad network connection, invalid file, etc.



A column was specified that doesn't exists.



A column was specified that doesn't exists.



So, it looks like something is missing or my changes are not being mapped across the shapefile components. I can't share the data but can share the code. The problem is with the renaming... What am I missing?



import geopandas as gp

# Read data in
f="my_file"
data = gp.read_file(f)

# Drop a column (if writing out file here, does as expected)
data=data.drop(['CLASS'], axis=1)

# Do the dissolve (if writing out file here, does as expected)
data=data.dissolve(by='Hazard_Pot', as_index=False)

# Rename (THE PROBLEM I THINK : if saving here, I get my issue in ArcMap)
data=data.rename(index=str, columns="Hazard_Pot":"Potential",
"casualties":"Casualty",
"vehicle_ac":"Access",
"building d":"Damage",
"disruption":"Disruption",
"contaminat":"Contam.",
"do_not_1":"Do_Not_1")

# Reorder
data=data[['Potential',
'Casualty',
'Access',
'Damage',
'Disruption',
'Contam.',
'Do_Not_1',
'geometry']]

# write to file (creating a new CRS based on the old as per https://gis.stackexchange.com/questions/204201/geopandas-to-file-saves-geodataframe-without-coordinate-system?noredirect=1&lq=1)
prj_file = "%s.prj" %os.path.splitext(f)[0]
prj = [l.strip() for l in open(prj_file,'r')][0]
data.to_file(ofile, driver=driver, crs_wkt=prj)






python shapefile geopandas






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 4 hours ago









ChrisWillsChrisWills

506




506












  • data.rename(index=str... renaming the index to a data type is very strange. Take that part out and report back

    – Paul H
    4 hours ago











  • I can't share the data but can share the code and you can share fake data that reproduces the issue

    – Paul H
    4 hours ago











  • @PaulH i belive index=str is correct syntax, see the examples in help section of rename. But I also think that part is not needed.

    – BERA
    4 hours ago












  • @BERA Oops - forgot rename could take callables.

    – Paul H
    3 hours ago

















  • data.rename(index=str... renaming the index to a data type is very strange. Take that part out and report back

    – Paul H
    4 hours ago











  • I can't share the data but can share the code and you can share fake data that reproduces the issue

    – Paul H
    4 hours ago











  • @PaulH i belive index=str is correct syntax, see the examples in help section of rename. But I also think that part is not needed.

    – BERA
    4 hours ago












  • @BERA Oops - forgot rename could take callables.

    – Paul H
    3 hours ago
















data.rename(index=str... renaming the index to a data type is very strange. Take that part out and report back

– Paul H
4 hours ago





data.rename(index=str... renaming the index to a data type is very strange. Take that part out and report back

– Paul H
4 hours ago













I can't share the data but can share the code and you can share fake data that reproduces the issue

– Paul H
4 hours ago





I can't share the data but can share the code and you can share fake data that reproduces the issue

– Paul H
4 hours ago













@PaulH i belive index=str is correct syntax, see the examples in help section of rename. But I also think that part is not needed.

– BERA
4 hours ago






@PaulH i belive index=str is correct syntax, see the examples in help section of rename. But I also think that part is not needed.

– BERA
4 hours ago














@BERA Oops - forgot rename could take callables.

– Paul H
3 hours ago





@BERA Oops - forgot rename could take callables.

– Paul H
3 hours ago










1 Answer
1






active

oldest

votes


















3














Looking at your code I can see two things that Arcmap doesn't like:



  • The first one is the space in the field name "building d";

  • The second one - and the cause of the error - is the . in "contaminat":"Contam.".

I was able to reproduce the error by creating a test file and using your code. Removing the dot "contaminat":"Contam" solved the problem.






share|improve this answer








New contributor




Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.



















    Your Answer








    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "79"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f316876%2frenaming-columns-using-geopandas-issues-with-data-table-in-arc%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    3














    Looking at your code I can see two things that Arcmap doesn't like:



    • The first one is the space in the field name "building d";

    • The second one - and the cause of the error - is the . in "contaminat":"Contam.".

    I was able to reproduce the error by creating a test file and using your code. Removing the dot "contaminat":"Contam" solved the problem.






    share|improve this answer








    New contributor




    Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
    Check out our Code of Conduct.
























      3














      Looking at your code I can see two things that Arcmap doesn't like:



      • The first one is the space in the field name "building d";

      • The second one - and the cause of the error - is the . in "contaminat":"Contam.".

      I was able to reproduce the error by creating a test file and using your code. Removing the dot "contaminat":"Contam" solved the problem.






      share|improve this answer








      New contributor




      Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
      Check out our Code of Conduct.






















        3












        3








        3







        Looking at your code I can see two things that Arcmap doesn't like:



        • The first one is the space in the field name "building d";

        • The second one - and the cause of the error - is the . in "contaminat":"Contam.".

        I was able to reproduce the error by creating a test file and using your code. Removing the dot "contaminat":"Contam" solved the problem.






        share|improve this answer








        New contributor




        Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.










        Looking at your code I can see two things that Arcmap doesn't like:



        • The first one is the space in the field name "building d";

        • The second one - and the cause of the error - is the . in "contaminat":"Contam.".

        I was able to reproduce the error by creating a test file and using your code. Removing the dot "contaminat":"Contam" solved the problem.







        share|improve this answer








        New contributor




        Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        share|improve this answer



        share|improve this answer






        New contributor




        Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.









        answered 4 hours ago









        Mouad AlamiMouad Alami

        894




        894




        New contributor




        Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.





        New contributor





        Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.






        Mouad Alami is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
        Check out our Code of Conduct.



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Geographic Information Systems Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid …


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fgis.stackexchange.com%2fquestions%2f316876%2frenaming-columns-using-geopandas-issues-with-data-table-in-arc%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Which organization defines CJK Unified Ideographs? The Next CEO of Stack OverflowCharacters which have several different shapesHow useful are the kanji in reading Chinese?Can Chinese readers scan large amounts of text faster/more accurately than their alphabet-using counterparts?丼: why is “well” also “bowl of food”?What Does Unicode 8.0 Mean For Chinese?How are blanks indicated for placeholders in Chinese (like ???)Is there a dictionary of standard character variants?How to display CJK Extension F?How is it decided as to which character is used on the tech terminology?How does 子 come to mean 'midnight'?

            Shenzhen Football Club Índice Elenco atual | Títulos | Referências Ligações externas | Menu de navegaçãoGooglenotíciaslivrosacadêmicoeditar«"Elenco"»Site oficialexpandindo-oeee

            When We Were Young (canção de Adele) Índice Antecedentes e lançamento | Faixas e formatos | Performances e covers | Desempenho nas tabelas musicais | Créditos | Histórico de lançamento | Referências Menu de navegação«Best albums of 2015»«Adele - When We Were Young (Radio Date: 22-01-2016)»«When We Were Young - Single by Adele»«Adele: Inside Her Private Life and Triumphant Return»«adele interview: world exclusive first interview in three years»«Tobias Jesso Jr: since Adele tweeted his song, he's even bigger than his dad»«Adele interviews Tobias Jesso Jr: 'I think that a couple of the ideas we had could be rap songs'»«Adele on Her Return: 'I Was So Frightened That No One Cared'»«Tobias Jesso Jr. on working with Adele: 'I was as nervous as shit'»«How Ariel Rechtshaid Pushed Adele To Her Limit»«Adele Previews 'When We Were Young' on '60 Minutes' Teaser, Tops Trending 140»«Adele Performs New '25' Ballad, "When We Were Young" Live: Watch»«Which '25' Song Should Be Adele's Next Single?»«Adele's 'When We Were Young' Confirmed As Second Single From '25'»«Adele's new single artwork for 'When We Were Young' is perfectly adorable»«Adele at the BBC review: honest, funny and spectacular – Celebrity News News»«'Saturday Night Live': Adele Sings 'Hello' and 'When We Were Young'»«'Adele: Live in New York City' NBC Special – Set List Revealed!»«Adele Closes Out the 2016 Brit Awards With 'When We Were Young'»«What Is The Adele Live Tour Set List? There's No Way She'd Leave Out These 8 Songs»«See Demi Lovato's Soaring Cover of Adele's 'When We Were Young'»«'The Voice': 5 Best Moments From Week 1 Blind Auditions»«Adele – When We Were Young (Media Control Charts)»«Adele – When We Were Young (Entertainment Monitoring Africa)»«Adele – When We Were Young (ARIA Charts)»«Adele – When We Were Young (Ö3 Austria Top 40)»«Adele – When We Were Young (Ultratop 50)»«Adele – When We Were Young (Ultratop 40)»«Adele – When We Were Young (Canadian Hot 100)»«Adele – When We Were Young (Tracklisten)»«Adele – When We Were Young (The Official Charts Company)»«Adele – Hello (IFPI Slovenská Republika)»«Adele – When We Were Young (Productores de Música de España)»«Adele – When We Were Young (Billboard Hot 100)»«Adele – When We Were Young (Pop Songs)»«Adele – When We Were Young (Adult Pop Songs)»«Adele – When We Were Young (Hot Adult Contemporary Charts)»«Adele – When We Were Young (Hot Dance Club Songs)»«Adele – When We Were Young (Rock Airplay)»«Adele – When We Were Young (IFPI Finlândia)»«Adele – When We Were Young (Syndicat National de l'Éditon Phonographique)»«Adele – When We Were Young (Magyar Hanglemezkiadók Szövetsége)»«Adele – When We Were Young (Irish Recorded Music Association)»«Adele – When We Were Young (Mexico Airplay)»«Adele – When We Were Young (VG-lista)»«Adele – When We Were Young (NZ Top 40 Singles)»«Adele – When We Were Young (MegaCharts)»«Adele – When We Were Young (Związek Producentów Audio Video)»«Adele – When We Were Young (Portugal Digital Songs)»«Adele – When We Were Young (UK Indie Singles Chart)»«Adele – When We Were Young (UK Singles Chart)»«Adele – When We Were Young (Sverigetopplistan)»«Adele – When We Were Young (Schweizer Hitparade)»«Adele – When We Were Young (Portugal Digital Songs)»«Music Canada – Gold/Platinum – When We Were Young»«NZ Top 40 Singles Chart»«Certified Awards»e