Good Afternoon,
As a user for route planning, I find the resource as it is, extremely useful.
However, I have a slightly odd request. I am working on a home project (running onboard on a Raspberry Pi) that tracks us travelling around the UK in our Narrowboat, taking photos, temperature and GPS location.
What I would like to do, is store the number of locks between 2 locations. Naturally I could store this by some sort of push button, or text input, but as CanalPlan has such a comprehensive database, I was hoping to utilise this already brilliant resource.
I have found that I can post Long/Lat to gazetteer, for example:
http://canalplan.org.uk/cgi-bin/gazetteer.cgi?where=52.442799333+-1.077916
This returns Google Suggestions, in my testing first result is always correct, so that is perfect for my use, and can automate the interrogate of the HTML to get the actual place name...
What I then want to be able to do is post to named locations to canal.cgi, to give me the route plan, which includes the number of locks.
However I am struggling to post the info to canal.cgi, in the way that it will plan the route.
What I would ideally like to be able to do is post:
http://canalplan.org.uk/cgi-bin/canal.cgi?pl_start=North Kilworth Narrows&pl_end=Husband's Bosworth Tunnel (Northeast end)
And the route be planned.
I can then read off the lock value from the planned route.
Any pointers you could give me would be brilliant.
Many Thanks
Greg
How much of the route planning functionality do you need? If you are just going to need the distance and number of locks between two places and you need the place names close to a lat / long then the data set used by the Canalplan plugin for Wordpress may suffice as it holds the lat and long of all canalplan places and all the links between places (which includes distance and number of locks)
If you store it in a MySQL database then you can easily do geographical queries on it.
Steve
CanalPlan has a bit more of an API than is obvious, it's just entirely undocumented and only used internally!
So, for example, to get a place from coordinates you can use:
http://canalplan.org.uk/cgi-bin/api.cgi?mode=nearest_place&lat=52.442799333&lng=-1.077916
This returns a block of data about the place in JSON format.
That should be easier to parse than page scraping and less load at our end!
Note that this returns a different place than the top Google suggestion, but looking at the database I think it's a better match to the actual lat/lng than the one you are getting.
There actually is a way to calculate the route - your guess wasn't very far off - using the following:
http://canalplan.org.uk/cgi-bin/canal.cgi?quickroute=yes&where=north%20kilworth%20narrows,husband's%20Bosworth%20Tunnel%20(Northeast%20end) (http://canalplan.org.uk/cgi-bin/canal.cgi?quickroute=yes&where=north%20kilworth%20narrows,husbands%20Bosworth%20Tunnel%20(Northeast%20end))
- this is how the search box on the left-hand menu works. For programmatic use you can improve this by using the canalplan IDs (returned from the api call - prefix them by a $ to distinguish from names):
http://canalplan.org.uk/cgi-bin/canal.cgi?quickroute=yes&where=$4283,$jjkq
But that still involves quite a lot of HTML scraping when you get the results page.
There isn't yet a \"show the route between two places\" option to the api but I can add it quite easily. I suggest just:
http://canalplan.org.uk/cgi-bin/api.cgi?mode=route&start=4283&end=jjkq
If this works for you I'll add it in the next couple of days. A JSON block with a list of steps and the totals already worked out should be pretty easy.
JSON!!!!! :-o absolutely perfect. If you were able to implement it easily that would be fabulous (either way you will have to let me have a paypal address to send some beer money to)... but I should now be able to get what I am after via HTML.
I attach a basic code sample below, which may help others around the getting location with the JSON API (C# using Mono on Linux).
Thanks so much,
Greg
public String GetCanalPlanLocation(String strLongitude, String strLatitude)
{
String strRetValue = \"Unknown\";
String strURL = null;
try
{
strURL = \"http://canalplan.org.uk/cgi-bin/api.cgi?mode=nearest_place&lat=\" + strLatitude + \"&lng=\" + strLongitude;
Console.WriteLine(\"GetCanalPlanLocation: \" + strURL);
String result = new System.Net.WebClient().DownloadString(strURL);
System.Web.Script.Serialization.JavaScriptSerializer jss = new System.Web.Script.Serialization.JavaScriptSerializer();
CanalPlan.Response dCPResponse = jss.Deserialize(result);
if (dCPResponse != null)
{
if (dCPResponse.mode == \"success\")
{
if((dCPResponse.name != null) && (dCPResponse.name != \"\")){ strRetValue = dCPResponse.name; }
}
else
{
Console.WriteLine(\"CanalPlan Status: \" + dCPResponse.mode);
}
}
dCPResponse = null;
}
catch (Exception ex)
{
Console.WriteLine(\"GetCanalPlanLocation: \" + ex.Message);
}
return strRetValue;
}
}
public class CanalPlan
{
public class Response
{
public String name { get; set; }
public String mode { get; set; }
public String canal { get; set; }
}
}
Yeah, JSON is great. Will work up that route calculation API for you soonest.
Paypal on home page is beer money address!
It took about 10 minutes, so I pushed it live.
The URL you need is:
http://canalplan.org.uk/cgi-bin/api.cgi?mode=plan&start=4283&end=jjkq
Those are the IDs you get back from the nearest_place call.
There are three items in the top level JSON: \"links\" is a list of each leg of the route, \"totals\" is a very high level summary and \"places\" is all the detail to map an ID into a place (name, size, attributes, coordinates etc) - this is to save you needing API calls to translate each of the links into English.
For sheer curiosity value, this is the code that generates it:
define function Plan_Route(cg as lookup)
database input system.config.main_data
variable toret as lookup = {links="", totals={distance=0, locks=0},places={}}
for i=route "'$'+cg.start,'$'+cg.end",'dot'
toret.links += i
toret.totals.distance += i.distance
toret.totals.locks += i.locks
for j=each "'place1','place2'"
if typeof(toret['places'][i[j]]) = 'string'
toret['places'][i[j]] = placeinfo('$'+i[j])
end if
end for j
end for i
return toret
end define
If you want any other items in the totals I can easily accumulate them.
Note that all distances are in metres. \"gcdist\" is the great-circle distance between the two places. Any queries on any of the other fields just ask!
I can't thank you enough - beer money incoming.
Using your API accumulating a day trip at a time, I am pleased to report we have bashed our way through 267 locks in the past 60 days!
Logic below on the off chance it helps others in the future.
Once again, many thanks for all your help!
Greg
public Int32 GetCanalPlanLocksBetween(String strStartLongitude, String strStartLatitude, String strEndLongitude, String strEndLatitude)
{
Int32 strRetValue = 0;
String strURL = null;
try
{
String strStartID = GetCanalPlanLocationID(strStartLongitude, strStartLatitude);
String strEndID = GetCanalPlanLocationID(strEndLongitude, strEndLatitude);
if ((strStartID != null) && (strEndID != null))
{
strURL = \"http://canalplan.org.uk/cgi-bin/api.cgi?mode=plan&start=\" + strStartID + \"&end=\" + strEndID;
String result = new System.Net.WebClient().DownloadString(strURL);
System.Web.Script.Serialization.JavaScriptSerializer jss = new System.Web.Script.Serialization.JavaScriptSerializer();
CanalPlan.Response dCPResponse = jss.Deserialize(result);
if (dCPResponse != null)
{
if (dCPResponse.totals != null)
{
strRetValue = dCPResponse.totals.locks;
}
}
dCPResponse = null;
}
}
catch (Exception ex)
{
WriteLog(\"GetCanalPlanLocation: \" + strURL + \": \" + ex.Message);
}
return strRetValue;
}
public String GetCanalPlanLocationID(String strLongitude, String strLatitude)
{
String strRetValue = null;
try
{
String strURL = \"http://canalplan.org.uk/cgi-bin/api.cgi?mode=nearest_place&lat=\" + strLatitude + \"&lng=\" + strLongitude;
String result = new System.Net.WebClient().DownloadString(strURL);
System.Web.Script.Serialization.JavaScriptSerializer jss = new System.Web.Script.Serialization.JavaScriptSerializer();
CanalPlan.Response dCPResponse = jss.Deserialize(result);
if (dCPResponse != null)
{
if (dCPResponse.mode == \"success\")
{
if ((dCPResponse.id != null) && (dCPResponse.id != \"\"))
{
strRetValue = dCPResponse.id;
}
}
}
dCPResponse = null;
}
catch (Exception ex)
{
WriteLog(\"GetCanalPlanLocationID: \" + ex.Message);
}
return strRetValue;
}
}
public class CanalPlan
{
public class Response
{
public String name { get; set; }
public String mode { get; set; }
public String id { get; set; }
public String canal { get; set; }
public Totals totals { get; set; }
}
public class Totals
{
public Int32 locks { get; set; }
public Int32 distance { get; set; }
}
}
A work in progress...
(https://s4.postimg.org/5nvlrm0ah/canalmaps.png) (https://postimg.org/image/5nvlrm0ah/)
So how often do you grab the GPS and call Canalplan? Just wondering because if you have too many places between the start end end points of the plan route then you could find there's more than one possible route.
I do like the screen grab - that's a LOT of photos!
Steve
Yes, the code as provided will always provide the \"shortest\" route - this (goes off and checks the source) - just counts 4 locks as one mile and then does the shortest path.
This reminds me of something I tried to do in the early days of CanalPlan using a laptop, a USB connected GPS and a webcam. In that case I was trying to establish the locations of bridges etc by looking at the snapshots and matching to the grid references. I could never really get it to work: running both over the serial port was more than the laptop could cope with. Also GPS was a lot worse than today's devices, even stuck on the roof it kept losing track.
I played with the idea of using some sort of AI to detect locks: there's probably a fairly distinct pattern of stopping, a few feet back and forth, then going, particularly when accompanied by some change in altitude. But as I never really got the basic tracking working I never took this any further.
So it grabs the GPS location every 20 seconds when the boat is moving more than 1mph or every 15 mins when idle (and takes a photo for a pretty sped up stop capture for each day)... saves the data locally to MySQL on the Pi.
Runs 24/7 as is very low voltage.
Then at the end of each day (or when internet is available via 3G dongle - often on a garden cane stuck in a log on the roof), I run a function that draws the map for the day and geo tags the location of each pic (using google api), then send just the start and end location to Canal Plan (so 1 query a day moving forward) for the lock count.
The only slight issue I noticed (which is a rare occurrence only when we have had visitors), is if we repeat a lock by turning around, sending start and end doesn't factor this in, as I haven't told Canal Plan what we did in between...
Or for example you get to the end of the Oxford Canal, and go down Isis Lock, turn around and come back up in the same trip - I have just told Canal Plan my start and end location was before the lock, so it doesn't count it - but that is a very rare occurrence, and can just overtype when this happens.
Attached was my prototype before we set off, its in a nice box now... switch and LEDs wired to the back of the boat so I can see its working. The switch (ON) off (ON), more of a button, for taking a photo/video on demand (and photos in tunnels where GPS drops off and doesn't know if we are moving or not).
(https://s9.postimg.org/qv3k2jphn/piproto.jpg) (https://postimg.org/image/qv3k2jphn/)
To get round that you'd need to send start, turn round and end point I guess.
One thing I have noticed (when playing with tracking - which Canalplan does ) was that if the GPS signal goes on a phone then it uses the mast locations so you tend up end up with some very odd bits in the route when you plot it unless you try to do some clever \"this point is too far from the last point so ignore it\" sort of logic
Quote from: Nicholas AttyIt took about 10 minutes, so I pushed it live.
The URL you need is:
http://canalplan.org.uk/cgi-bin/api.cgi?mode=plan&start=4283&end=jjkq
This and the rest of the JSON based API has proven to be really useful to me.
I'm in the process of beginning a boat moving business and being able to offer a quote from a web page based on lock miles is a great boon.
I had to shunt the calls to canalplan through my own server's python based CGI to get around CORS security issues, but that was the only really tricky part.
My tool:
http://yourhelmsman.aegidian.org/trip.htmlPlease let me know how best to attribute the back-end to Canalplan, and the beer is already waiting.
I suspect a \"Data provided by Canalplan\" with a link back to canalplan.org.uk would be enough.....
Easily done. Again, thank you.
Hi, I am currently planning a big trip from London to the North West and Canalplan is extremely helpful. I would like to throw together all my planning information in QGIS using shapefiles from CRT and OpenStreetMap.
It would be nice to also use information from the Canal Plan Gazetteer. Are you also able to make this information available through a JSON API. I am particularly interested in the mooring quality field.
Out of interest I cannot find any licence information on your website. Reading the post above it sounds like you provide on something like Creative Commons-Attribution or are you more restrictive?
There's already a rough-and-ready API call to do that:
https://canalplan.eu/cgi-bin/api.cgi?mode=place&id=xxxx will return the information in JSON format where xxxx is a canalplan ID. So, for example, https://canalplan.eu/cgi-bin/api.cgi?mode=place&id=bu4s returns the info for the A6 Road Bridge in Bedford:
[{\"osx\":\"505075\",\"type\":\"2\",\"mouth\":\"\",\"changeline\":\"\",\"mooring_JSON\":{\"type\":\"0\",\"rating\":\"40\",\"detail\":\"Moorings for town by park.\"},\"name\":\"A6 Road Bridge (Bedford)\",\"roadname\":\"\",\"lock_JSON\":{},\"latitude\":\"52.1345384341288\",\"osy\":\"249580\",\"misc\":\"\",\"hlng\":\"0W27'58\\\"\",\"detail\":\"\",\"sunset\":\"4:58 PM\",\"winding_JSON\":{},\"hlat\":\"52N8'4\\\"\",\"graphic\":\"h1\",\"areas\":\"\",\"osgrid\":\"TL050495\",\"postcode\":\"MK42 0AR\",\"spur\":\"\",\"longitude\":\"-0.466232299804687\",\"access_JSON\":{\"verified\":\"yes\",\"show\":\"noshow\",\"access\":\"no\"},\"sunrise\":\"7:34 AM\",\"roadnumber\":\"\",\"attributes\":\"Z=\",\"id\":\"bu4s\",\"bridge_JSON\":{\"crossing\":\"0\",\"under\":false,\"missing\":false},\"navnote\":\"\"}]
Mooring information is in mooring_JSON. This is all very poorly, if at all, documented but if you can make intelligent guesses you're free to use it for your own purposes for the moment.
This database extract should help you decode the \"type\" and \"rating\" for the mooring_JSON:
INSERT INTO \"mooring_types\" VALUES(0,0,'unrated','no information available');
INSERT INTO \"mooring_types\" VALUES(0,10,'impossible','don''t try it!');
INSERT INTO \"mooring_types\" VALUES(0,20,'tolerable','it''s just about possible if really necessary');
INSERT INTO \"mooring_types\" VALUES(0,30,'ok','a perfectly adequate mooring');
INSERT INTO \"mooring_types\" VALUES(0,40,'good','a nice place to moor');
INSERT INTO \"mooring_types\" VALUES(0,50,'excellent','this is a really good mooring');
INSERT INTO \"mooring_types\" VALUES(1,0,'unrated','no information available');
INSERT INTO \"mooring_types\" VALUES(1,10,'pins','mooring pins are needed');
INSERT INTO \"mooring_types\" VALUES(1,20,'piling','piling suitable for hooks');
INSERT INTO \"mooring_types\" VALUES(1,30,'rings','mooring rings or bollards are available');
0=ratings and 1=types
I need to work out hard how to license this (it will be suitably permissive, but it does cost server load without the advertising revenue we get from normal use) and indeed all the data and code.
Note that the mooring info is still pretty patchy - it's the main reason I've not yet coded anything to use it into the route planner.
Quote from: AegidianEasily done. Again, thank you.
That looks great. I do reserve the right at some stage to be slightly more restrictive in terms of use at some stage in the future, but you've earned a fair amount of credit with contributions to the database for quite some time yet!
Thank you for the details of the JSON. Very helpful.
Hopefully we can contribute to the database on our way up.
Posting this in an existing API thread because it seemed to be relevant to earlier questions:
- so do start a separate thread if more appropriate
Forgive me if this has been answered as I don't know where to look:
Love CanalPlanAC and staring to think how to use it on a daily basis.
Is there a Gazetteer API to choose the sections to be displayed, instead of using preference settings?
I am setting up a route spreadsheet for our helmsperson with hyperlinks to the Gazetteer URLs
e.g. https://canalplan.org.uk/place/tghr etc.
I would like to set the sections to be displayed as shown in the Gazetteer configuration list:
Information
Photos
Comments
Map
Blogs
Geograph
etc.
etc.
Linking directly from a spreadsheet may mean that the users browser is not logged onto the CanalPlanAC website so preferences will not be applied, and we don't want all the photos, google search and other links.
I don't need the JSON interface as I am quite happy to display the page returned by the Gazetteer.
I was hoping for something like:
https://canalplan.org.uk/place/<4charaterid>?display=\"some (https://canalplan.org.uk/place/%3C4charaterid%3E?display=some) string defining the list of fields\"
Thanks Laurence
That would be quite a serious change to the url handling code so I suspect it wouldn't happen quickly
Thanks for the quick response.
I don't think it would be too difficult to add an interface so you can say \"link to this place as displayed\" - feel free to drop it into the issue tracker as a new idea.
I'm futzing around in PHP trying to get the 'mode=plan' section of the API to return a route that will accept exclusions (particularly the Tidal Severn, waterway id: mc07.)
I've been digging around and for some reason recall that you can specify pref=options when building the API call, however I can't seem to stumble on the right format for 'options'.
I've tried sending plaintext pref = 'optcheck_wex_mc07', and pref 'wex_mc07' and an associated array of values pref = { optcheck_wex_mc07 : true }
// fetch POST routine
//
// exclude optcheck_wex_m0c7 - Tidal River Severn
//
$url = 'https://canalplan.uk/cgi-bin/api.cgi';
$opts = ['optcheck_wex_m0c7' => true];
$opts = 'optcheck_wex_m0c7';
$opts = ['wex' => 'm0c7'];
$opts = 'wex_mc07';
//$opts = '';
$data = ['mode' => 'plan', 'start' => $place1['id'], 'end' => $place2['id'] , 'pref' => $opts];
var_dump($data);
// use key 'http' even if you send the request to https://...
$options = [
'http' => [
'header' => \"Content-type: application/x-www-form-urlencoded\\r\\n\",
'method' => 'POST',
'content' => http_build_query($data),
],
];
$context = stream_context_create($options);
$result = file_get_contents($url, false, $context);
var_dump($result);
How should I be building this part of the query?
That's a good question - I think this needs nick. I think it may well be a JSON structure you need to pass in