[DISCUSSION] Thought process on how to add elevation

[albertkun]

Hello! GIS Specialist here! Great work on this project!!

I Just wanted to start a discussion on how to achieve elevation with the OpenStreetMap (OSM) data set in to Arnis, since I see it is on the todo list!

I believe the most suitable format would have to be Digital Elevation Model (DEM) data or some other raster input.

I'd be happy to help out with implementing this, but wanted to see what the current thought process for achieving this would be! I haven't coded anything Minecraft related before, but I work extensively with GIS data and enjoy coding in Rust!

Additional thoughts

• I think implementing the scaling factor in the current Rust version would be helpful for this (which I saw from [FEATURE] Scale feature  #49 that it was implemented in Python)
• I think the process could be:
  1. Grab OSM data from Overpass API from a Bounding Box
  2. Grab DEM data for the same Bounding Box
  3. Map the elevation in Minecraft
  4. Add the OSM data to the elevation in Minecraft

[louis-e]

Hi there! That's amazing, thank you for reaching out! Really cool to get feedback and input from people who actually know what they're doing. ;)

I had a pretty similar thought process. I already played around in the "old" Python version with implementing some sort of elevation, but I failed with getting a data source which is:

1. free
2. preferably doesn't need an API token
3. has a large coverage of mainly the US and Europe
4. is purely the ground level (filtered out tree and building heights)

In regards of the implementation, like you mentioned we'd have to map the elevation data to the exact same area as the elements using the bbox and the scaling factor. I'm pretty sure this is just some maybe annoying fine tuning work, but sounds doable. For the next step, my approach would have been to use the position as an index when placing blocks to retrieve the according elevation at that exact coordinate. But this assumes that we have a data source which fits our criterias. I can imagine some pretty weird structures if there's e.g. a sudden peak in elevation due to an unfiltered tree height, if you know what I mean.

I will use your Issue as a discussion thread for further updates on this topic. Thanks for the DEM hint, I will look into this further. Curious to hear your thoughts on this! :)

[albertkun]

Thanks for the detailed and supportive reply!!

In terms of free APIs we could potentially utilize:

• https://portal.opentopography.org/apidocs/

But I think users would need to sign up with an API key and we'd have to parameterize that as a command line argument.

Open Topography

• Endpoint: https://portal.opentopography.org/API/globaldem

• Query parameters:

  parameter	description	example
  demtype	Dataset type	"SRTMGL1", "SRTMGL3", "AW3D30", "NASADEM"
  south, north, west, east	Bounding box coordinates	41.0, 42.0, -72.0, -71.0
  outputFormat	Output format	"GTiff" for GeoTIFF

• Request format:

    params = {
        "demtype": dem_type,
        "south": south,
        "north": north,
        "west": west,
        "east": east,
        "outputFormat": "GTiff"
    }

Not sure if there is another free solution without API keys, but I'll keep looking around!

[ArmokGoB]

Another issue is the world height limit. I could stand in the ocean and end up 960 meters above sea level with 8 hours of walking where I used to live.

[tumluliu]

Hi, another GISer here :) Very nice work and I'm thinking about this same topic when exploring the generated world.

This open-sourced elevation API project open-elevation could be a good candidate but it's not active any more unfortunately. Some of its forks, e.g. this one according to this ticket, are well maintained.

There is a free public API at https://open-elevation.com/ but the availability could not be guaranteed. The self-hosted service is encouraged and easy to setup. Since I guess the elevation support in Arnis will be optional, to have a simple script launching open-elevation locally when required could be a feasible solution.

[DanHues]

Another issue is the world height limit. I could stand in the ocean and end up 960 meters above sea level with 8 hours of walking where I used to live.

Hey on the note of height limit, in the latest releases of mc in world data packs you can set the height limit to a max of 2000+ so this part shouldn’t be too much of an issue

[scd31]

Hi, another GISer here :) Very nice work and I'm thinking about this same topic when exploring the generated world.

This open-sourced elevation API project open-elevation could be a good candidate but it's not active any more unfortunately. Some of its forks, e.g. this one according to this ticket, are well maintained.

There is a free public API at https://open-elevation.com/ but the availability could not be guaranteed. The self-hosted service is encouraged and easy to setup. Since I guess the elevation support in Arnis will be optional, to have a simple script launching open-elevation locally when required could be a feasible solution.

Open Elevation is, afaik, just a wrapper around the SRTM dataset. We could use this dataset directly if we wanted, without requiring the wrapper, however it's captured from satellites so I'd be worried about that messing it up. I suppose it would be relatively easy to check the dataset to see if this is an issue or not.

[scd31]

Here's WIP elevation (:

[faithsapling]

Hi there! That's amazing, thank you for reaching out! Really cool to get feedback and input from people who actually know what they're doing. ;)

I had a pretty similar thought process. I already played around in the "old" Python version with implementing some sort of elevation, but I failed with getting a data source which is:

1. free
2. preferably doesn't need an API token
3. has a large coverage of mainly the US and Europe
4. is purely the ground level (filtered out tree and building heights)

In regards of the implementation, like you mentioned we'd have to map the elevation data to the exact same area as the elements using the bbox and the scaling factor. I'm pretty sure this is just some maybe annoying fine tuning work, but sounds doable. For the next step, my approach would have been to use the position as an index when placing blocks to retrieve the according elevation at that exact coordinate. But this assumes that we have a data source which fits our criterias. I can imagine some pretty weird structures if there's e.g. a sudden peak in elevation due to an unfiltered tree height, if you know what I mean.

I will use your Issue as a discussion thread for further updates on this topic. Thanks for the DEM hint, I will look into this further. Curious to hear your thoughts on this! :)

Some years ago, a modder named Gegy made an earth mod with elevation called Terrarium. It can be found here on github, perhaps it might give you some tips

[carterlasalle]

Hi there! That's amazing, thank you for reaching out! Really cool to get feedback and input from people who actually know what they're doing. ;)
I had a pretty similar thought process. I already played around in the "old" Python version with implementing some sort of elevation, but I failed with getting a data source which is:

1. free
2. preferably doesn't need an API token
3. has a large coverage of mainly the US and Europe
4. is purely the ground level (filtered out tree and building heights)

In regards of the implementation, like you mentioned we'd have to map the elevation data to the exact same area as the elements using the bbox and the scaling factor. I'm pretty sure this is just some maybe annoying fine tuning work, but sounds doable. For the next step, my approach would have been to use the position as an index when placing blocks to retrieve the according elevation at that exact coordinate. But this assumes that we have a data source which fits our criterias. I can imagine some pretty weird structures if there's e.g. a sudden peak in elevation due to an unfiltered tree height, if you know what I mean.
I will use your Issue as a discussion thread for further updates on this topic. Thanks for the DEM hint, I will look into this further. Curious to hear your thoughts on this! :)

Some years ago, a modder named Gegy made an earth mod with elevation called Terrarium. It can be found here on github, perhaps it might give you some tips

https://www.gebco.net/data_and_products/gridded_bathymetry_data/

"GEBCO’s current gridded bathymetric data set, the GEBCO_2024 Grid, is a global terrain model for ocean and land, providing elevation data, in meters, on a 15 arc-second interval grid. It is accompanied by a Type Identifier (TID) Grid that gives information on the types of source data that the GEBCO_2024 Grid is based on. "

@louis-e LMK if you need any testing, I'm on MacOS.

[carterlasalle]

Another issue is the world height limit. I could stand in the ocean and end up 960 meters above sea level with 8 hours of walking where I used to live.

You can allways cap it, or restrict the world height by injecting a datapack:

{ "type": "minecraft:overworld", "generator": { "type": "minecraft:noise", "settings": { "min_y": -64, "height": 512 } } }

[faithsapling]

Another earth to Minecraft tool called Chunkmapper by whamtet has tackled the elevation issue by simply only generating maps at 1:30 scale so that Mount Everest would within the height limit. Of course, he didn't use the earth data for real life buildings because they'd be way too small, so he had Chunkmapper generate rows and columns of random buildings for cities. Oh and another thing about that tool is that it's meant to keep on running so that it would continue to generate more chunks based on earth data as you move around in game.

Now suppose we didn't worry too much about the higher elevations on earth and just focus on having the tallest building in the world to scale down to fit within minecraft's height limit, we could have the option to have earth generated at 1:3 scale (33.33%).

[Kingsyss]

@scd31 How did you make the landscape?

[scd31]

@Kingsyss See #180

[louis-e]

scd31 provided us his amazing work which we are now continuing to develop in the elevation-wip branch. I already resolved the merge conflicts to the current main code base. The next thing we have to do is adapting all element processors (src/element_processing/*) to the new feature and then replace the current sine wave implementation with an actual elevation data parser.

[carterlasalle]

scd31 provided us his amazing work which we are now continuing to develop in the elevation-wip branch. I already resolved the merge conflicts to the current main code base. The next thing we have to do is adapting all element processors (src/element_processing/*) to the new feature and then replace the current sine wave implementation with an actual elevation data parser.

I have tried out f50e108 and I know it is not finished, but its cool seeing the progress being made. Right now it looks like basic sin wave and there are random circular holes, but I cant wait.

[carterlasalle]

@louis-e @albertkun I did some research and had AI just format it but there are some good sources below

Suitable Heightmap Data Sources for Arnis

• Free to use: No cost associated with accessing and using the data.
• Preferably no API token required: To simplify the user experience.
• Large coverage: Including at least the US and Europe.
• Digital Terrain Model (DTM): Representing the bare earth elevation, with vegetation and buildings removed as much as possible.

Candidate Data Sources:

1. NASA's SRTM (Shuttle Radar Topography Mission)

   • Pros:
     • Freely available.
     • Global coverage (between 60°N and 60°S), including most of the US and Europe.
     • Resolution of 1 arc-second (about 30 meters) or 3 arc-seconds (about 90 meters).
     • Generally considered a DTM in most areas, although some vegetation and building artifacts may remain.
     • Available in GeoTIFF format.
     • No API key required for direct download.
   • Cons:
     • Resolution might be a bit coarse for highly detailed city generation in Minecraft.
     • Data gaps ("voids") exist in some areas (can be filled using interpolation or other datasets).
   • Access:
     • USGS EarthExplorer: https://earthexplorer.usgs.gov/
     • NASA's Earthdata Search: https://search.earthdata.nasa.gov/
     • OpenTopography: https://portal.opentopography.org/ (API key may be required for some functionalities, but direct download of SRTM data is possible)

2. ASTER Global Digital Elevation Model (ASTER GDEM)

   • Pros:
     • Freely available.
     • Global coverage (83°N to 83°S).
     • Resolution of 1 arc-second (about 30 meters).
     • Available in GeoTIFF format.
     • No API key required for direct download.
   • Cons:
     • Can have more noise and artifacts compared to SRTM, especially in steep terrain or cloudy areas.
     • Requires registration to download from some sources.
   • Access:
     • NASA's Earthdata Search: https://search.earthdata.nasa.gov/
     • USGS EarthExplorer: https://earthexplorer.usgs.gov/

3. Open-Elevation API

   • Pros:
     • Free and open-source API.
     • Provides elevation data for any point on Earth.
     • Combines data from multiple sources, including SRTM and ASTER, to fill gaps and potentially improve accuracy.
   • Cons:
     • Public API is rate-limited and availability is not guaranteed long-term.
     • Resolution depends on the underlying data sources, which can vary.
     • Data quality might not be consistent across different regions.
   • Access:
     • Public API: https://open-elevation.com/
     • Self-hosting is recommended for heavy use.
     • GitHub: https://github.com/Jorl17/open-elevation

4. Copernicus DEM

   • Pros:
     • Free and open access.
     • Global coverage at various resolutions.
     • The GLO-30 DTM has a resolution of 1 arc-second, comparable to SRTM.
     • Effort towards making it a bare earth product (although some artifacts might still remain).
   • Cons:
     • GLO-90 has a resolution of 3 arc-seconds.
     • May require registration to access through some platforms.
   • Access:
     • Airbus: https://www.intelligence-airbusds.com/en/8694-copernicus-dem-access
     • Amazon Web Services (S3): https://registry.opendata.aws/copernicus-dem/
     • European Space Agency (ESA) Copernicus Open Access Hub: https://scihub.copernicus.eu/

[albertkun]

thank you @carterlasalle !!

so exciting to see this progress!!! let me know if there's any need for support on geospatial processing of the data in the elevation-wip branch!!

[louis-e]

I now ported every element processor to the new ground level handling in the elevation-wip branch.
Also I'm close to getting a proof of concept working using the Mapbox Terrain-RGB API.

[louis-e]

Progress! Finally managed to map the elevation data to the generation map. Will push it soon to the elevation-wip branch when I cleaned up the code a bit and fixed some bugs!

[albertkun]

Looks beautiful!!!

[louis-e]

Sorry for the delay, catching up with all the progress on main requires more time than expected haha! Also vacation is over since last week and I need to work again. I'm freezing the main branch for a few days until the terrain is fully implemented, then I'll get back to all the pull requests!

[louis-e]

Merged into main! :)
Make sure to check the 'Terrain' checkbox in the generation settings to enable it!

I want to test it extensively in the next couple of days before creating a release with it. Feel free to drop some feedback here and let me know if this feature should be enabled by default. I'll also have to catch up with a few pull requests :)

[carterlasalle]

Merged into main! :) Make sure to check the 'Terrain' checkbox in the generation settings to enable it!

I want to test it extensively in the next couple of days before creating a release with it. Feel free to drop some feedback here and let me know if this feature should be enabled by default. I'll also have to catch up with a few pull requests :)

Hi! Im loving playing around with it so far! I think it should eventually be added as default, but not now. For example, one area that had houses in the past releases, now doesn't

!

[carterlasalle]

Wait, never mind. It is correct. @louis-e I did not realize that OSM updated that quickly. My house and the entire neighborhood burned down a couple of weeks ago in the massive Los Angeles fires. Disregard the above!

[louis-e]

Oh no, I'm so sorry for your loss. Didn't know the building data was already removed on OSM... @carterlasalle

[albertkun]

this is amazing!! i'll probably get around to testing it next week! i fled the fires from Los Angeles and am visiting my family at the moment :( but excited to test it when i get home!!

[albertkun]

okay! did some testing and maybe this is more of a problem with the OSM data (because it happens in other platforms too using 3d buildings and OSM data), but some buildings happen to cut the terrain out instead of being placed on top of it. i believe this is related to this #325 (comment)..

Example

1 .5 scale

Example 2

1.20 scale

I think what we usually try to do to solve this is to extrude the foundation of the building(1) so that it is level where (2) is. Hope that makes sense!