scgraph

scgraph

PyPI version License: MIT PyPI Downloads

Supply chain graph package for Python

scgraph

Documentation

Getting Started: https://github.com/connor-makowski/scgraph

Low Level: https://connor-makowski.github.io/scgraph/scgraph/core.html

Key Features

  • Calculate the shortest path between two points on earth using a latitude / longitude pair
    • Inputs:
      • A latitude / longitude pair for the origin
      • A latitude / longitude pair for the destination
    • Calculation:
      • Algorithms:
        • Dijkstra's algorithm (Modified for sparse networks)
          • Modified to support sparse network data structures
        • Makowski's Modified Sparse Dijkstra algorithm
          • Modified for O(n) performance on particularly sparse networks
        • A* algorithm (Extension of Makowski's Modified Sparse Dijkstra)
          • Uses a heuristic function to improve performance on large graphs
            • Note: The heuristic function is optional and defaults to Dijkstra's algorithm
        • Possible future support for other algorithms
      • Distances:
    • Returns:
      • path:
        • A list of dictionaries (latitude and longitude) that make up the shortest path
      • length:
        • The distance in kilometers between the two points
  • Antimeridian support
  • Arbitrary start and end points
  • Arbitrary network data sets
  • Grid based graphs
  • Cached shortest path calculations for very fast repetative calculations to or from the same point in a graph.
    • Note: Geographs are not yet supported for this feature

Setup

Make sure you have Python 3.10.x (or higher) installed on your system. You can download it here.

Note: Support for python3.6-python3.9 is available up to version 2.2.0.

Installation

pip install scgraph

Use with Google Colab

Getting Started

Basic Usage

Get the shortest path between two points on earth using a latitude / longitude pair In this case, calculate the shortest maritime path between Shanghai, China and Savannah, Georgia, USA.

# Use a maritime network geograph
from scgraph.geographs.marnet import marnet_geograph

# Get the shortest maritime path between Shanghai, China and Savannah, Georgia, USA
# Note: The origin and destination nodes can be any latitude / longitude pair
output = marnet_geograph.get_shortest_path(
    origin_node={"latitude": 31.23,"longitude": 121.47},
    destination_node={"latitude": 32.08,"longitude": -81.09},
    output_units='km'
)
print('Length: ',output['length']) #=> Length:  19596.4653

In the above example, the output variable is a dictionary with three keys: length and coordinate_path.

  • length: The distance between the passed origin and destination when traversing the graph along the shortest path
    • Notes:
      • This will be in the units specified by the output_units parameter.
      • output_units options:
        • km (kilometers - default)
        • m (meters)
        • mi (miles)
        • ft (feet)
  • coordinate_path: A list of lists [latitude,longitude] that make up the shortest path

For more examples including viewing the output on a map, see the example notebook.

Included GeoGraphs

  • marnet_geograph:
    • What: A maritime network data set from searoute
    • Use: from scgraph.geographs.marnet import marnet_geograph
    • Attribution: searoute
    • Length Measurement: Kilometers
    • Marnet Picture
  • oak_ridge_maritime_geograph:
  • north_america_rail_geograph:
  • us_freeway_geograph:
  • scgraph_data geographs:
    • What: Additional geographs are available in the scgraph_data package
      • Note: These include larger geographs like the world highways geograph and world railways geograph.
    • Installation: pip install scgraph_data
    • Use: from scgraph_data.world_highways import world_highways_geograph
    • See: scgraph_data for more information and all available geographs.

GridGraph usage

Example:

  • Create a grid of 20x100 cells.
    • This creates a grid based graph with connections to all 8 neighbors for each grid item.
    • Each grid item has 4 cardinal connections at length 1 and 4 diagonal connections at length sqrt(2)
  • Create a wall from (10,5) to (10,99).
    • This would foce any path to go to the bottom of the graph to get around the wall.
  • Get the shortest path between (2,10) and (18,10)
    • Note: The length of this path should be 16 without the wall and 20.9704 with the wall.
from scgraph import GridGraph

x_size = 20
y_size = 20
blocks = [(10, i) for i in range(5, y_size)]

# Create the GridGraph object
gridGraph = GridGraph(
    x_size=x_size,
    y_size=y_size,
    blocks=blocks,
    add_exterior_walls=True,
)

# Solve the shortest path between two points
output = gridGraph.get_shortest_path(
    origin_node={"x": 2, "y": 10},
    destination_node={"x": 18, "y": 10},
    # Optional: Specify the output coodinate format (default is 'list_of_dicts)
    output_coordinate_path="list_of_lists",
    # Optional: Cache the origin point spanning_tree for faster calculations on future calls
    cache=True,
    # Optional: Specify the node to cache the spanning tree for (default is the origin node)
    # Note: This first call will be slower, but future calls using this origin node will be substantially faster
    cache_for="origin",
)

print(output)
#=> {'length': 20.9704, 'coordinate_path': [[2, 10], [3, 9], [4, 8], [5, 8], [6, 7], [7, 6], [8, 5], [9, 4], [10, 4], [11, 4], [12, 5], [13, 6], [14, 7], [15, 7], [16, 8], [17, 9], [18, 10]]}

Advanced Usage

Using scgraph_data geographs:

  • Note: Make sure to install the scgraph_data package before using these geographs
from scgraph_data.world_railways import world_railways_geograph
from scgraph import Graph

# Get the shortest path between Kalamazoo Michigan and Detroit Michigan by Train
output = world_railways_geograph.get_shortest_path(
    origin_node={"latitude": 42.29,"longitude": -85.58},
    destination_node={"latitude": 42.33,"longitude": -83.05},
    # Optional: Use the A* algorithm
    algorithm_fn=Graph.a_star,
    # Optional: Pass the haversine function as the heuristic function to the A* algorithm
    algorithm_kwargs={"heuristic_fn": world_railways_geograph.haversine},
)

Get a geojson line path of an output for easy visualization:

  • Note: mapshaper.org and geojson.io are good tools for visualizing geojson files
from scgraph.geographs.marnet import marnet_geograph
from scgraph.utils import get_line_path

 # Get the shortest sea path between Sri Lanka and Somalia
output = marnet_geograph.get_shortest_path(
    origin_node={"latitude": 7.87,"longitude": 80.77},
    destination_node={"latitude": 5.15,"longitude": 46.20}
)
# Write the output to a geojson file
get_line_path(output, filename='output.geojson')

Modify an existing geograph: See the notebook here

You can specify your own custom graphs for direct access to the solving algorithms. This requires the use of the low level Graph class

from scgraph import Graph

# Define an arbitrary graph
# See the graph definitions here:
# https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph
graph = [
    {1: 5, 2: 1},
    {0: 5, 2: 2, 3: 1},
    {0: 1, 1: 2, 3: 4, 4: 8},
    {1: 1, 2: 4, 4: 3, 5: 6},
    {2: 8, 3: 3},
    {3: 6}
]

# Optional: Validate your graph
Graph.validate_graph(graph=graph)

# Get the shortest path between idx 0 and idx 5
output = Graph.dijkstra_makowski(graph=graph, origin_id=0, destination_id=5)
#=> {'path': [0, 2, 1, 3, 5], 'length': 10}

You can also use a slightly higher level GeoGraph class to work with latitude / longitude pairs

from scgraph import GeoGraph

# Define nodes
# See the nodes definitions here:
# https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph
nodes = [
    # London
    [51.5074, -0.1278],
    # Paris
    [48.8566, 2.3522],
    # Berlin
    [52.5200, 13.4050],
    # Rome
    [41.9028, 12.4964],
    # Madrid
    [40.4168, -3.7038],
    # Lisbon
    [38.7223, -9.1393]
]
# Define a graph
# See the graph definitions here:
# https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph
graph = [
    # From London
    {
        # To Paris
        1: 311,
    },
    # From Paris
    {
        # To London
        0: 311,
        # To Berlin
        2: 878,
        # To Rome
        3: 1439,
        # To Madrid
        4: 1053
    },
    # From Berlin
    {
        # To Paris
        1: 878,
        # To Rome
        3: 1181,
    },
    # From Rome
    {
        # To Paris
        1: 1439,
        # To Berlin
        2: 1181,
    },
    # From Madrid
    {
        # To Paris
        1: 1053,
        # To Lisbon
        5: 623
    },
    # From Lisbon
    {
        # To Madrid
        4: 623
    }
]

# Create a GeoGraph object
my_geograph = GeoGraph(nodes=nodes, graph=graph)

# Optional: Validate your graph
my_geograph.validate_graph()

# Optional: Validate your nodes
my_geograph.validate_nodes()

# Get the shortest path between two points
# In this case, Birmingham England and Zaragoza Spain
# Since Birmingham and Zaragoza are not in the graph,
# the algorithm will add them into the graph.
# See: https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph.get_shortest_path
# Expected output would be to go from
# Birmingham -> London -> Paris -> Madrid -> Zaragoza

output = my_geograph.get_shortest_path(
    # Birmingham England
    origin_node = {'latitude': 52.4862, 'longitude': -1.8904},
    # Zaragoza Spain
    destination_node = {'latitude': 41.6488, 'longitude': -0.8891}
)
print(output)
# {
#     'length': 1799.4323,
#     'coordinate_path': [
#         [52.4862, -1.8904],
#         [51.5074, -0.1278],
#         [48.8566, 2.3522],
#         [40.4168, -3.7038],
#         [41.6488, -0.8891]
#     ]
# }

Development

Running Tests, Prettifying Code, and Updating Docs

Make sure Docker is installed and running on a Unix system (Linux, MacOS, WSL2).

  • Create a docker container and drop into a shell
    • ./run.sh
  • Run all tests (see ./utils/test.sh)
    • ./run.sh test
  • Prettify the code (see ./utils/prettify.sh)
    • ./run.sh prettify
  • Update the docs (see ./utils/docs.sh)

    • ./run.sh docs
  • Note: You can and should modify the Dockerfile to test different python versions.

Attributions and Thanks

Originally inspired by searoute including the use of one of their datasets that has been modified to work properly with this package.

  1"""
  2# scgraph
  3[![PyPI version](https://badge.fury.io/py/scgraph.svg)](https://badge.fury.io/py/scgraph)
  4[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
  5[![PyPI Downloads](https://img.shields.io/pypi/dm/scgraph.svg?label=PyPI%20downloads)](https://pypi.org/project/scgraph/)
  6
  7Supply chain graph package for Python
  8
  9
 10![scgraph](https://raw.githubusercontent.com/connor-makowski/scgraph/main/static/scgraph.png)
 11
 12## Documentation
 13
 14Getting Started: https://github.com/connor-makowski/scgraph
 15
 16Low Level: https://connor-makowski.github.io/scgraph/scgraph/core.html
 17
 18
 19## Key Features
 20
 21- Calculate the shortest path between two points on earth using a latitude / longitude pair
 22    - Inputs:
 23        - A latitude / longitude pair for the origin
 24        - A latitude / longitude pair for the destination
 25    - Calculation:
 26        - Algorithms:
 27            - Dijkstra's algorithm (Modified for sparse networks)
 28                - Modified to support sparse network data structures
 29            - Makowski's Modified Sparse Dijkstra algorithm
 30                - Modified for O(n) performance on particularly sparse networks
 31            - A* algorithm (Extension of Makowski's Modified Sparse Dijkstra)
 32                - Uses a heuristic function to improve performance on large graphs
 33                    - Note: The heuristic function is optional and defaults to Dijkstra's algorithm
 34            - Possible future support for other algorithms
 35        - Distances:
 36            - Uses the [haversine formula](https://en.wikipedia.org/wiki/Haversine_formula) to calculate the distance between two points on earth
 37    - Returns:
 38        - `path`:
 39            - A list of dictionaries (`latitude` and `longitude`) that make up the shortest path
 40        - `length`:
 41            - The distance in kilometers between the two points
 42- Antimeridian support
 43- Arbitrary start and end points
 44- Arbitrary network data sets
 45- Grid based graphs
 46- Cached shortest path calculations for very fast repetative calculations to or from the same point in a graph.
 47    - Note: Geographs are not yet supported for this feature
 48
 49
 50## Setup
 51
 52Make sure you have Python 3.10.x (or higher) installed on your system. You can download it [here](https://www.python.org/downloads/).
 53
 54Note: Support for python3.6-python3.9 is available up to version 2.2.0.
 55
 56## Installation
 57
 58```
 59pip install scgraph
 60```
 61
 62## Use with Google Colab
 63
 64- [Getting Started](https://colab.research.google.com/github/connor-makowski/scgraph/blob/main/examples/getting_started.ipynb)
 65- [Creating A Multi Path Geojson](https://colab.research.google.com/github/connor-makowski/scgraph/blob/main/examples/multi_path_geojson.ipynb)
 66- [Modifying A Geograph](https://colab.research.google.com/github/connor-makowski/scgraph/blob/main/examples/geograph_modifications.ipynb)
 67
 68
 69# Getting Started
 70
 71## Basic Usage
 72
 73Get the shortest path between two points on earth using a latitude / longitude pair
 74In this case, calculate the shortest maritime path between Shanghai, China and Savannah, Georgia, USA.
 75
 76```py
 77# Use a maritime network geograph
 78from scgraph.geographs.marnet import marnet_geograph
 79
 80# Get the shortest maritime path between Shanghai, China and Savannah, Georgia, USA
 81# Note: The origin and destination nodes can be any latitude / longitude pair
 82output = marnet_geograph.get_shortest_path(
 83    origin_node={"latitude": 31.23,"longitude": 121.47},
 84    destination_node={"latitude": 32.08,"longitude": -81.09},
 85    output_units='km'
 86)
 87print('Length: ',output['length']) #=> Length:  19596.4653
 88```
 89
 90In the above example, the `output` variable is a dictionary with three keys: `length` and `coordinate_path`.
 91
 92- `length`: The distance between the passed origin and destination when traversing the graph along the shortest path
 93    - Notes:
 94        - This will be in the units specified by the `output_units` parameter.
 95        - `output_units` options:
 96            - `km` (kilometers - default)
 97            - `m` (meters)
 98            - `mi` (miles)
 99            - `ft` (feet)
100- `coordinate_path`: A list of lists [`latitude`,`longitude`] that make up the shortest path
101
102For more examples including viewing the output on a map, see the [example notebook](https://colab.research.google.com/github/connor-makowski/scgraph/blob/main/examples/getting_started.ipynb).
103
104## Included GeoGraphs
105
106- marnet_geograph:
107    - What: A maritime network data set from searoute
108    - Use: `from scgraph.geographs.marnet import marnet_geograph`
109    - Attribution: [searoute](https://github.com/genthalili/searoute-py)
110    - Length Measurement: Kilometers
111    - [Marnet Picture](https://raw.githubusercontent.com/connor-makowski/scgraph/main/static/marnet.png)
112- oak_ridge_maritime_geograph:
113    - What: A maritime data set from the Oak Ridge National Laboratory campus
114    - Use: `from scgraph.geographs.oak_ridge_maritime import oak_ridge_maritime_geograph`
115    - Attribution: [Oak Ridge National Laboratory](https://www.ornl.gov/) with data from [Geocommons](http://geocommons.com/datasets?id=25)
116    - Length Measurement: Kilometers
117    - [Oak Ridge Maritime Picture](https://raw.githubusercontent.com/connor-makowski/scgraph/main/static/oak_ridge_maritime.png)
118- north_america_rail_geograph:
119    - What: Class 1 Rail network for North America
120    - Use: `from scgraph.geographs.north_america_rail import north_america_rail_geograph`
121    - Attribution: [U.S. Department of Transportation: ArcGIS Online](https://geodata.bts.gov/datasets/usdot::north-american-rail-network-lines-class-i-freight-railroads-view/about)
122    - Length Measurement: Kilometers
123    - [North America Rail Picture](https://raw.githubusercontent.com/connor-makowski/scgraph/main/static/north_america_rail.png)
124- us_freeway_geograph:
125    - What: Freeway network for the United States
126    - Use: `from scgraph.geographs.us_freeway import us_freeway_geograph`
127    - Attribution: [U.S. Department of Transportation: ArcGIS Online](https://hub.arcgis.com/datasets/esri::usa-freeway-system-over-1500k/about)
128    - Length Measurement: Kilometers
129    - [US Freeway Picture](https://raw.githubusercontent.com/connor-makowski/scgraph/main/static/us_freeway.png)
130- `scgraph_data` geographs:
131    - What: Additional geographs are available in the `scgraph_data` package
132        - Note: These include larger geographs like the world highways geograph and world railways geograph.
133    - Installation: `pip install scgraph_data`
134    - Use: `from scgraph_data.world_highways import world_highways_geograph`
135    - See: [scgraph_data](https://github.com/connor-makowski/scgraph_data) for more information and all available geographs.
136
137## GridGraph usage
138
139Example:
140- Create a grid of 20x100 cells.
141    - This creates a grid based graph with connections to all 8 neighbors for each grid item.
142    - Each grid item has 4 cardinal connections at length 1 and 4 diagonal connections at length sqrt(2)
143- Create a wall from (10,5) to (10,99).
144    - This would foce any path to go to the bottom of the graph to get around the wall.
145- Get the shortest path between (2,10) and (18,10)
146    - Note: The length of this path should be 16 without the wall and 20.9704 with the wall.
147
148```py
149from scgraph import GridGraph
150
151x_size = 20
152y_size = 20
153blocks = [(10, i) for i in range(5, y_size)]
154
155# Create the GridGraph object
156gridGraph = GridGraph(
157    x_size=x_size,
158    y_size=y_size,
159    blocks=blocks,
160    add_exterior_walls=True,
161)
162
163# Solve the shortest path between two points
164output = gridGraph.get_shortest_path(
165    origin_node={"x": 2, "y": 10},
166    destination_node={"x": 18, "y": 10},
167    # Optional: Specify the output coodinate format (default is 'list_of_dicts)
168    output_coordinate_path="list_of_lists",
169    # Optional: Cache the origin point spanning_tree for faster calculations on future calls
170    cache=True,
171    # Optional: Specify the node to cache the spanning tree for (default is the origin node)
172    # Note: This first call will be slower, but future calls using this origin node will be substantially faster
173    cache_for="origin",
174)
175
176print(output)
177#=> {'length': 20.9704, 'coordinate_path': [[2, 10], [3, 9], [4, 8], [5, 8], [6, 7], [7, 6], [8, 5], [9, 4], [10, 4], [11, 4], [12, 5], [13, 6], [14, 7], [15, 7], [16, 8], [17, 9], [18, 10]]}
178```
179
180## Advanced Usage
181
182Using `scgraph_data` geographs:
183- Note: Make sure to install the `scgraph_data` package before using these geographs
184```py
185from scgraph_data.world_railways import world_railways_geograph
186from scgraph import Graph
187
188# Get the shortest path between Kalamazoo Michigan and Detroit Michigan by Train
189output = world_railways_geograph.get_shortest_path(
190    origin_node={"latitude": 42.29,"longitude": -85.58},
191    destination_node={"latitude": 42.33,"longitude": -83.05},
192    # Optional: Use the A* algorithm
193    algorithm_fn=Graph.a_star,
194    # Optional: Pass the haversine function as the heuristic function to the A* algorithm
195    algorithm_kwargs={"heuristic_fn": world_railways_geograph.haversine},
196)
197```
198
199Get a geojson line path of an output for easy visualization:
200- Note: `mapshaper.org` and `geojson.io` are good tools for visualizing geojson files
201```py
202from scgraph.geographs.marnet import marnet_geograph
203from scgraph.utils import get_line_path
204
205 # Get the shortest sea path between Sri Lanka and Somalia
206output = marnet_geograph.get_shortest_path(
207    origin_node={"latitude": 7.87,"longitude": 80.77},
208    destination_node={"latitude": 5.15,"longitude": 46.20}
209)
210# Write the output to a geojson file
211get_line_path(output, filename='output.geojson')
212```
213
214Modify an existing geograph: See the notebook [here](https://colab.research.google.com/github/connor-makowski/scgraph/blob/main/examples/geograph_modifications.ipynb)
215
216
217You can specify your own custom graphs for direct access to the solving algorithms. This requires the use of the low level `Graph` class
218
219```py
220from scgraph import Graph
221
222# Define an arbitrary graph
223# See the graph definitions here:
224# https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph
225graph = [
226    {1: 5, 2: 1},
227    {0: 5, 2: 2, 3: 1},
228    {0: 1, 1: 2, 3: 4, 4: 8},
229    {1: 1, 2: 4, 4: 3, 5: 6},
230    {2: 8, 3: 3},
231    {3: 6}
232]
233
234# Optional: Validate your graph
235Graph.validate_graph(graph=graph)
236
237# Get the shortest path between idx 0 and idx 5
238output = Graph.dijkstra_makowski(graph=graph, origin_id=0, destination_id=5)
239#=> {'path': [0, 2, 1, 3, 5], 'length': 10}
240```
241
242You can also use a slightly higher level `GeoGraph` class to work with latitude / longitude pairs
243
244```py
245from scgraph import GeoGraph
246
247# Define nodes
248# See the nodes definitions here:
249# https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph
250nodes = [
251    # London
252    [51.5074, -0.1278],
253    # Paris
254    [48.8566, 2.3522],
255    # Berlin
256    [52.5200, 13.4050],
257    # Rome
258    [41.9028, 12.4964],
259    # Madrid
260    [40.4168, -3.7038],
261    # Lisbon
262    [38.7223, -9.1393]
263]
264# Define a graph
265# See the graph definitions here:
266# https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph
267graph = [
268    # From London
269    {
270        # To Paris
271        1: 311,
272    },
273    # From Paris
274    {
275        # To London
276        0: 311,
277        # To Berlin
278        2: 878,
279        # To Rome
280        3: 1439,
281        # To Madrid
282        4: 1053
283    },
284    # From Berlin
285    {
286        # To Paris
287        1: 878,
288        # To Rome
289        3: 1181,
290    },
291    # From Rome
292    {
293        # To Paris
294        1: 1439,
295        # To Berlin
296        2: 1181,
297    },
298    # From Madrid
299    {
300        # To Paris
301        1: 1053,
302        # To Lisbon
303        5: 623
304    },
305    # From Lisbon
306    {
307        # To Madrid
308        4: 623
309    }
310]
311
312# Create a GeoGraph object
313my_geograph = GeoGraph(nodes=nodes, graph=graph)
314
315# Optional: Validate your graph
316my_geograph.validate_graph()
317
318# Optional: Validate your nodes
319my_geograph.validate_nodes()
320
321# Get the shortest path between two points
322# In this case, Birmingham England and Zaragoza Spain
323# Since Birmingham and Zaragoza are not in the graph,
324# the algorithm will add them into the graph.
325# See: https://connor-makowski.github.io/scgraph/scgraph/core.html#GeoGraph.get_shortest_path
326# Expected output would be to go from
327# Birmingham -> London -> Paris -> Madrid -> Zaragoza
328
329output = my_geograph.get_shortest_path(
330    # Birmingham England
331    origin_node = {'latitude': 52.4862, 'longitude': -1.8904},
332    # Zaragoza Spain
333    destination_node = {'latitude': 41.6488, 'longitude': -0.8891}
334)
335print(output)
336# {
337#     'length': 1799.4323,
338#     'coordinate_path': [
339#         [52.4862, -1.8904],
340#         [51.5074, -0.1278],
341#         [48.8566, 2.3522],
342#         [40.4168, -3.7038],
343#         [41.6488, -0.8891]
344#     ]
345# }
346
347```
348
349# Development
350## Running Tests, Prettifying Code, and Updating Docs
351
352Make sure Docker is installed and running on a Unix system (Linux, MacOS, WSL2).
353
354- Create a docker container and drop into a shell
355    - `./run.sh`
356- Run all tests (see ./utils/test.sh)
357    - `./run.sh test`
358- Prettify the code (see ./utils/prettify.sh)
359    - `./run.sh prettify`
360- Update the docs (see ./utils/docs.sh)
361    - `./run.sh docs`
362
363- Note: You can and should modify the `Dockerfile` to test different python versions.
364
365
366## Attributions and Thanks
367Originally inspired by [searoute](https://github.com/genthalili/searoute-py) including the use of one of their [datasets](https://github.com/genthalili/searoute-py/blob/main/searoute/data/marnet_densified_v2_old.geojson) that has been modified to work properly with this package."""
368
369from .core import Graph, GeoGraph
370from .grid import GridGraph