All posts by Nick Martinelli

Layer Config with Leaflet

Code for the leaflet layer configuration script described in this post is available on GitHub here.

I currently have the pleasure of working with the  Mary’s River Watershed Council to create web maps for their in development website update. The map above shows the in progress map (No those aren’t the final colors).

One of the biggest issues with web maps for folks without developers on staff is that they can become out of date very quickly. One of the main challenges on this project is to make it easy to update for a non technical staff. I think that this solution, with a little documentation, provides a good model for making good dynamic maps for non-profits and other typically low-resource groups.

The Challenges

  • Map showing all projects within the council work area.
  • Maps on several pages (~12) to show different work areas and projects.
  • Still have access to show all other projects when desired.
  • Click/Hover interactivity from various field names depending on the data.
  • Do it on a shoestring.

And the big one

  • Data must be update-able by non developer GIS staff.

I considered using one of our usual suspects for quick web maps, like Mapbox and CartoBD (which are great by the way), but neither quite fit with the various maps the MRWC wanted to display, and the ultimate data update flexibility they desired.

So, custom solution here we come. I decided to create a map with Leaflet and add two things to make the maps very customizable, and up-dateable for the staff. First, add a custom CONFIG file, in JSON, so that it was pretty readable for the MRWC staff. Second, make the visible layers, map center, and map zoom settable through the URL Variables, so the embedded iFrames can be set to display exactly the layers and geography desired. With the config file, and URL control, we can make one map to rule them all!

The config.json

The config file specifies the following about all layers available in the map.

  • All overlay geojson layers.
  • Display name for each layer.
  • Paths to the geojson for each layer.
  • Style settings for each layer.
  • Geometry type for each layer. (limits the type of geojson you can use, but increases simplicity for non-technical staff use)
  • Hover and Click interactivity fields if any for each layer.

example config.json section for ‘Wetlands’ layer:

"displayname":"Wetlands",
"pathtofile":"resources/geojson/Wetlands071814.json",
"stroke":true,
"color":"#33a02c",
"weight":4,
"opacity":0.9,
"fill":true,
"fillColor":"#33a02c",
"fillOpacity":0.2,
"geometry":"polygon",
"hover":true,
"hoverArray":[{"displayname":"Project","field":"ProjectNam"}],
"click":true,
"clickArray":[{"displayname":"Project Name","field":"ProjectNam"},{"displayname":"Subbasin","field":"Subbasin"},{"displayname":"Square Feet","field":"sq_ft"},{"displayname":"Status","field":"Status"}]

URL Variables (really!)

URL variables are used to specify:

  • lyrAr*: A list of layers from configjson to display. Defaults to all layers visible if not specified.
  • lat and lng: Used to set map center. Defaults to center specified in script.js if not set.
  • zoom: The map zoom level. Defaults to zoom specified in script.js if not set.

This way, all layers are always available, but only those specified in the URL are displayed initially.

Screen Shot 2014-09-23 at 12.21.01 PM

A Couple Examples

The map at the top of this page shows vanilla map linked at www.nickmartinelli.com/mrwc.

This map is loaded as an iFrame with the URL variables of ‘lyrArr’ for just the MRWC administrative boundary and sub basins.  http://nickmartinelli.com/mrwc/?lyrArr=MRWC%20Boundary,SubBasins

This map is loaded as an iFrame with the URL variables of ‘lyrArr’ for the MRWC administrative boundary and sub basins and the Wetlands layer. It is also zoomed to the main wetlands work area with hover and click enabled for the Wetlands layer. http://nickmartinelli.com/mrwc/?lyrArr=MRWC%20Boundary,SubBasins,Wetlands&zoom=14&lat=44.593&lng=-123.415

So the MRWC can add as many maps as they want, using iFrames from one single map source. I think that this solution, produced quickly, will provide the flexibility MRWC desires and will be easy for them to update and modify as projects evolve.

As usual, the code is available on GitHub here.

There was a fire.

Homes+in+north+Corvallis+evacuated+by+fire+near+Chip+Ross+Park+-+Photo+from+Justin+Quinn+Photography+01
Photo from Justin Quinn Photography

Nobody was killed, not even injured. But it was a big fire right in town. As the smoke filled the air around my house (safely 2 miles from the fire) I started to wonder what was going on. I took to twitter to get some information, and it didn’t seem good.

Uncontained and in a residential area. So while reading I found a lot of info about size and location descriptions but no maps, nothing showing described evacuation areas and potential evacuation areas where folks should be getting ready to move. So I made a map. Here is a screen grab after updates the next morning.

Screen Shot 2014-09-17 at 11.19.49 AM

Red ‘suitcases’ meant evacuation order had been issued, orange meant prepare for evacuation order. These orders come from emergency services, but I was sourcing reports from the Gazette Times.

Here is the map link, but the image below is what you see when you browse to any of my Mapbox free account maps right now.

Screen Shot 2014-09-17 at 4.10.30 PM

And that is just fine. I hit the ceiling of some arbitrary number of map views that Mapbox uses to control free/paid subscriptions. Fair enough, no problem. The map was done by the time it was cut off. By done, I mean it no longer had value as a map. It was an ephemeral map. It served a purpose and went away.

icon_60768
Icon by Matt Brooks

The map was started on a whim, because I couldn’t find a map on the web of where it was burning. I tweeted this out.

While not overwhelming, the response was quick, and there were folks that appreciated a decent map to see current and potential evacuation areas. It really did amaze me just how many people were using twitter to get updates. I got several replies and private messages from the current, and update tweets made later that night.

The map was retweeted a small number of times, and then by some local TV news reporters and must have ended up posted somewhere that drove more traffic to the map. Notice that the Gazette Times is included in the above tweet. I included them in my initial tweet in hopes that they might retweet the map, as it seemed helpful. However, I can understand a news organization not wanting to put out unconfirmed info, it would be easy enough to confirm. Instead they produced their own map shown here.

540aa3759fe30.image
Gazette Times

Some of my initial concerns about producing a potentially inaccurate map were quickly abated when I viewed this local newspaper map. I figured I couldn’t be less accurate. I also chose to show potential evacuation zones, and updated status, any pertinent information, and always tied to a source. Usually the Gazette Times.

This story has a happy ending because no one was hurt and the fire crews did an amazing job protecting people and structures . But as you can see from the image below, the situation could have turned much worse very quickly. The homes are feet from the fire line.

540bdc445169f.image
Gazette Times

Information dissemination is critical in an emergency, and simple tools like Mapbox make it so easy to produce high quality maps.

But there is a big issue with citizen volunteers. What if my house was in the evacuation zone after I had started the project? How could someone else pick up the map baton? It made me think of a huge feature request for our friends at mapbox. Wouldn’t it be great if, instead of just the map viewer URL, we could share an editor URL for mapbox maps. This way, others, without much training, could use these powerful tools to make updates and provide valuable information.  So…

Also, I would like to invite any reporters from the Gazette Times to the first #maptime meetup in Corvallis by @maptimCVO. We will be exploring map making in a beginner friendly way, and it would be so easy for the GT to add nice maps of important information!

Play : Learn : Sasquatch

I made the above map to learn a few things and have a little fun. You can see the full version here. I made it because I wanted to figure out what to use to rotate markers to a bearing and animate markers along paths generated by an API. I used the TriMet API for Portland’s public transit vehicle locations, which is in beta according to the website. There are categories of markers, paths to follow of varying lengths, and bearings.

…and what if the markers where something like northwest monsters? That would be fun.

For the type of project described above, I try to make learning something new a part of the design. And then assemble the tools for the job.

It is easy to pick your tools first. Either because we are used to them, or they are comfortable, or they are what the cool kids are using, etc. I’m guilty of it too!

So many tweets can be boiled down to “look what I made with ‘x'”. And again, I’m guilty of it too!

I am almost always interested in the first 4 words of that sentence, the “Look what I made…” part. I always want to see what creative and interesting maps and visualizations people come up with.

As soon as you pick your tools, the possible outcomes are limited. But when you design first, without thinking about whether you can do it with tool ‘x’, the possibilities are only limited by imagination.

It is as much a mental exercise as anything. Often clients make certain tools required or a project needs to happen fast. In those cases we all just use our favorite hammer. But when building to learn I will always try to avoid opening my tool box first. I try to come up with something fun to make and then acquire the bits necessary to make it happen.

That said, half the shit I’ve done lately has been using the same tools.

So, the code for the monster map is here.

More Neighborhoods

neighborhoodsI presented about the neighborhood project at the NACIS conference in Greenville, SC in October of this year. I talked a little about how the site was then, but mostly pointed out what I wanted to improve. A new version is posted HERE. The major issues for me were:

  1. User contributed comments were not visible.
  2. The data was not easily accessible.
  3. The code was not open and available for folks to implement on their own.

Descriptions

vanport

Descriptions like the one above were collected in the old version, but couldn’t be accessed live. Obviously, someone put time and thought into their description here. It was a shame that these descriptions were not visible. But now they are! Here are some more of my favorites.

awesomevilleCapitolHill

CentralDistrict

That last one is revealing in many ways. There is something potentially problematic about the term ‘neighborhoods’ that was brought up at the NACIS presentation by Dennis McClendon referencing Chicago. He brought up that people often use their parish to describe where they live, and also asserted that the African-American community may give you an intersection when asked what area do you live in. Things to think about in future iterations, or if anyone makes their own version. I’m really looking forward to other comments that reveal perceptions or ideas of place that I was not aware of.

The Data

download

First, the data is available to download directly from the site. There is a new little toolbar on the right which has a download button. The data comes in shapefile format, and will include all geometries including those that have been flagged as junk or inappropriate. But, here is a Direct Link to the latest data. This will include everything, even items added just now thanks to cartDB’s ability to export directly.

The Code

code

The code for this project is freely available on GitHub. I would love to hear if you do anything with it. And also if you make any improvements. (There is plenty of room for improvement). Here is the Direct Link.

The Design and New Features

Online maps do not need to be ugly. You can create cartography online, and even in web Mercator. As you can see from the first screen shot, we are pretty much in greyscale. Thanks to Stamen Design’s Toner base, it was pretty easy to set up. I wanted color to only come from User Contibutions. So you won’t find color until you start contributing or viewing other contributions.

The colors  for highlighted neighborhoods was fun to create. You are limited to 30 highlighted hoods, which is much more than the 12 you could highlight in the old version. The colors come from a fantastic tool I found on the web for generating distinguishable palettes called I Want Hue. Not a replacement for Color Brewer, but a great option if you need a larger set of qualitative colors than CB will provide.

Highlighted hoods are shown with little inset maps of each version of the neighborhood. Additionally now you can Love and Flag neighborhood. I really just needed the flag functionality so the community could police itself, but that seemed to be too negative, so I added the love button to cheer things up a bit. You can see the flag and love buttons here.

swanislandAre you still reading?

OK, that is a lot. But some of it is important to why the site is what it is. I think we can have a truly beautiful cartographic conversation about place through tools like this. The cartography builds itself through contributions.  I know I’ve already learned some things about areas I’m not familiar with from reading neighborhood comments on the site. So I hope you join in the conversation or start one of your own with the code.

stillpretty

 

Opacity Stacks

Opacity is multiplicative!

For the Draw Your Neighborhoods site I wanted to allow contributors and anyone else interested to access instant feedback. Given that it was an off time project, and designed as a learning experience for myself with certain tools (Leaflet, cartoDB, etc.), I did not want to spend a lot of time, or any time really, mucking with the data in the short term. But I did want anyone who contributed to be able to get an instant view of their contribution. So I had to come up with something that could be automated and would respect individual contributions.

So I decided that I would simply allow overlapping transparency to guide the user towards areas of agreement or disagreement. Above is an example. The Wallingford neighborhood in Seattle has 4 versions drawn by various contributors, with a relatively high level of agreement. Using what I’ll call an opacity stack, you can pretty well understand where agreement and disagreement exist.

The process turned out to be a little more complex than I originally expected. The problem is as follows, given N neighborhoods, what opacity should be used to reach a total opacity of 0.8 (my arbitrary choice). Originally I thought this would mean using opacity with an additive blend mode and simply dividing 0.8 by N. Well, CartoCSS does not provide additive blending of opacity(Or didn’t, maybe it does now, these things move fast).

Opacity is multiplicative. Imagine light passing through windows that are 50% opaque. Two windows would not block all of the light, the first would block half of the light, then the second would block half of the remaining light. This is illustrated (poorly) below.

So how would the site come up with the appropriate opacity for a specific highlighted neighborhood?

The answer (hey kids, you will use it when you grow up): Algebra.

Doing a #lazyweb search for the formula, I found a great post here. It contains the basic formula for combined transparency. Which is fairly intuitive once you consider the natural behavior of light as outlined above.

 opacity_new = 1 - (1 - opacity_a) * (1 - opacity_b)

With this formula our problem becomes clear. We want opacity_new to always be 0.8. And we want that true for N elements. Which results in this formula:

0.8 = 1 - (1-x)^N

We want to solve for x where N is the number of instances of a selected neighborhood within the database. After refreshing my algebraic brain, with the help of my wife, Alicia Miao, the formula to solve for the oppacity(x) of N objects to add to an opacity of 0.8 is:

x = 1 - Nth root(.2)

The formula results in an interesting slope as N increases. The slope works well with what we are trying to show with the overlapping polygons. As more contributions are made it becomes harder to ‘move’ the neighborhood. Here is the opacity curve. And that background isn’t just some chart junk, it shows a gradient from opacity of 0 to 0.8.

This formula is plugged into the code that creates the CartoCSS that is used as the style for the tiles generated when you highlight a neighborhood. After getting the neighborhood name and the number of drawings that have been made of the specified neighborhood, the values are plugged into the formula, and CartoCSS is created.

var n = neighborhood.name;
var c = neighborhood.count;
var fillOpacity = 1-(Math.pow(.2,1/c));
style += "[name = '"+n+"']";
style += "{polygon-fill:"+userNeighborhoods[i].color+";polygon-opacity:"+fillOpacity+";}";

In the case of Wallingford in Seattle, N = 4. With an N of 4, the above formula finds that the opacity(x) should be 0.33126. Go ahead, plug it into the formulas above….

OK, now that we have that confirmed, we write it into CartoCSS, and ship that off to the CartoDB machine and some how get back pretty tiles. Just like that! This all happens when you click ‘View Maps’ and highlight a neighborhood.

Opacity stacks share some cartographic considerations with Value by Alpha mapping. Check it out here. While Value by Alpha represents well thought out cartographic exploration and experimentation, Opacity Stacks (oh, I capitalized it like it’s a thing!) represent a briefly considered, sloppily implemented, easy answer to real time visualization of overlapping polygons. What these techniques do share is the effective use of visual weight to represent some aspect of the data. In the neighborhood map, this is the number of contributions, and cumulatively, the agreement of contributions. Unfortunately, I just checked my copy of ‘Semiology’ and there was no mention of opacity or ‘weight’. I’ve gone down another cartographic cul-de-sac, and I’m just riding my Mongoose around the empty lot at the end of the block.

Personal flashbacks aside, I still recommend checking out the tools used for this project, including CartDB, CartoCSS, Leaflet. I hope to post some improvements to the site shortly, including availability of comments like the one here. “…But I don’t trust them”. I love it. And a link to download the data (From the public, to the public right?).

PROBLEMS

Lots, but here is one; Conflict creates conflict. The opacity stack formula is based on single neighborhoods, and gives a great view of agreement/disagreement for a single discreet neighborhood. It is carto crap when polygons of different neighborhoods overlap.

Each of the above Portland neighborhoods overlap, but have various drawing counts. ‘Kerns’ and ‘southeast’ are drawn once, while Laurelhurst has 2 versions. So their opacity is calculated only for their own internal N, and overlapping areas are muddy. This isn’t a big enough issue that I feel the need to fix it right away, but it could be really interesting to use some kind of comp-op in the new CartoDB 2.0 with expanded support for CartoCSS to create some blends when neighborhoods have conflict. And of course we could do some more sophisticated analysis of the data to create better maps, but I think the opacity stack works in a pinch.

Most of the maps are screen shots from Draw Your Neighborhood.

As always, your cartographic feedback is appreciated!

Neighborhoods Project – Portland, Seattle, Vancouver

This Project is an attempt to map neighborhoods as one piece of a city’s image. Neighborhoods can be generally defined as two-dimensional sections of a city that have a defined extent. They are areas which you can enter or exit or use to mentally orient within a city. Real and imagined, neighborhoods make a difference in every aspect of our lives. They are zones of inclusion and exclusion, used by municipalities, realtors, school districts, tourists and locals to define what is special or unique about a place or even how you can interact with a space. Neighborhoods are defined by all of the people moving in, out and through them.

The project lives at pnwmaps.com/neighborhoods.

The site was built to allow you to map neighborhoods in Portland, Seattle, and Vancouver, BC.

Neighborhood names are not defined. It will be great to watch as neighborhoods develop and overlap from user contributions. Especially the way areas of agreement and conflict, inclusion and exclusion emerge from your contributions.

There are a few questions once you submit your neighborhood, but the only required one is ‘Neighborhood Name’. Simply select from that list as you type if the name exists already, or add your own. 

Once you add your neighborhood (or hopefully many neighborhoods), check out the live results! Click ‘View Maps’ to see what has been contributed so far. Here is a screen shots from test data.

Toggle the base map and Highlight Neighborhoods for a pretty map of the geometry of neighborhoods;

An opacity formula is used to generate the appropriate opacity for n contributions so the full opacity of overlapping polygons of the same neighborhood is always .8. CartoDB is being used to save and generate result tiles. It was great to work with and I recommend giving it a look. In addition, this site was built using Leaflet, Bootstrap, CartoDB, cartoCSS. Mapquest Tiles, and more. I’ll go over technical details in another post.

If you know Portland, Seattle, or Vancouver, BC — Please Contribute!

I also want to point out that this project took considerable inspiration from Bostonography’s ‘Mapping Boston’s Neighborhoods‘. If you can contribute there, go for it.

  “…we must learn to see the hidden forms in the vast sprawl of our cities” – The Image of The City, Lynch 1960

Portland Bridges Update

You can see the old version of the bridge map in this Previous Post. Here are some images from the new map. It now has all 11 bridges across the Willamette in Portland. The goal here was to be incredibly simple, highlight the bridge locations and profiles, and show how people can get across them. I think it gets there, and as always feedback and criticisms are welcome.

It is of course free to download as a full resolution PDF HERE. If you would rather order a print, you can do that HERE. Half of any profits made on these maps will go to Friends of Trees! Look for their signs when you are in Portland or Eugene. I also plan to bring a stack to give away at NACIS next week. The prints were done by Green Solutions Printing in Eugene, and I think they did a great job. I love the smell of the printer in the morning!