Maker Metrics: Measuring the Impact of Academic Makerspaces

We know that academic makerspaces create meaningful experiences for students, yet how do we measure this meaningful impact? Where do we start?
Last fall on behalf of the UVM FabLab I attended the first International Symposium on Academic Makerspaces (ISAM) at MIT. Organizers for the symposium came from MIT, Yale, Stanford, Olin, Carnegie Mellon, Case Western Reserve, Georgia Tech and UC Berkeley.
I specifically attended the conference to learn how to better use metrics to measure the impact of academic makerspaces. The big idea emphasized by Malcom Cooke of ThinkBox at Case Western was  Keep Calm and Collect Data – collect as much data as possible and start today! Collect data that you can assess and measure over time, look to metrics and outcomes, quantitative and qualitative outcomes.
Here are 10 suggestions for how to compile and share academic makerspace metrics.

1. Customize your makerspace story to different audiences. School administrators, current students, prospective students, potential donors and parents to name a few. Use data to back up the different stories you tell. Talking points could include:

• Student engagement, classes and workshops in the space.
• Recruitment
• Retention
• Research support
• Cool and exciting projects

2. Track usage of your space on a daily basis. ThinkBox at Case Western has a staffed welcome desk that requires sign in. In the UVM FabLab, a much smaller space than ThinkBox, our projects are tracked through an online ticket system called Footprints. The software is licensed by UVM and available to UVM entities to use at no cost. The ticket system is managed by student techs and serves as a queue for machine use while capturing student net ids, course numbers and names of professors overseeing the projects. Other technology available for capturing usage data include:

• Ipad sign-in apps
• Ticketing systems that your institutions might already license. UVM uses Footprints and we started using it to track FabLab projects two years ago.
• Checking photo ids and driver’s licenses and collecting information from them.
• Break beam sensors and counters. Build your own!

3. Track who uses your space. Even if you want to keep it simple and just track student ids and course numbers, your registrar’s office or the like can run this information for you and provide demographics like gender and major. If you want to take more information you can identify academic department usage and your different audiences such as students, faculty, and visitors.

4. Don’t forget to track special events and tours. Keep track of special events and guests to your space. For example, prospective students, visiting educators, local businesses, local and state government, etc. For a prospective student tour, it might not be practical to get information about individuals, but you can track more broadly. For example, you can track how many prospective students and family members participated in an open house event that included a tour of your space.

5. Track how the space is being used. Based on usage what machines are getting the most use? This could inform future purchasing decisions. Also consider asking your users what tools they would like access to that you don’t currently have. Use this information to assess the future machine makeup of your space.

• Student class projects
• Research support
• Creation of teaching tools
• Personal projects
• Technical training
• Curriculum integration
• Prototyping for startups

6. Identify how your makerspace plays a role in student recruitment and retention. Is your space a deciding factor when students enroll at your institution? How do tours of your space support student recruitment and how does access to your space support student retention? Survey your students to find out.

7. Compare how your makerspace is used in comparison to other facilities in your institution. Compare your numbers with those of other labs and facilities like the library, gym and other places that encourage learning outside the classroom.

8. Pay attention to your space’s demographics. Across the board makerspaces are used more frequently by white males than other groups. The Mechanical Engineering Department at the University of Delaware has noticed that through assigning individual projects (rather than group projects) more female students are engaging in the use hands-on tools and technologies. Does the diversity balance of your school match that of your space? By assessing this information you’ll be able to identify how welcoming your space is to different groups at your school.

9. What are your students learning in your makerspace that they aren’t learning in a traditional classroom? What do students get out of working in your space that they don’t get elsewhere? Are they gaining 21st century skills? What do you want your students to learn and is that what they are learning? Survey your students to find out. Georgia Tech is in the process of such a study
There are a few other metrics I that think are worth looking into. These weren’t directly addressed in the metrics sessions at the symposium but they are on my mind:

Remember to keep track of the events you hold in your academic makerspace.

Let’s pay attention to the environmental impact of makerspaces, this is also a meaningful metric.
The consumables we use at the UVM FabLab like pressed board and colored PLA and ABS plastics are very difficult to impossible to recycled locally and in some instances aren’t recycled anywhere. We have the potential to deepen the innovation potential of makerspaces if we can manage our plastic waste. If our bad print runs end up in the Great Pacific Garbage Patch that’s not innovation that’s just a sad everyday story. There are some interesting prospects out there for turning ABS and PLA printing waste into recycled filament, least of which is Filabot, a company founded in Vermont that sells grinders and spoolers. We’re looking into this and will follow up in a future post.
Are students who use makerspaces during their education more employable when they graduate?
If students are learning key skills like those identified in this Forbes article about what employers want in graduates, are educational makerspaces making our students more employable? From my experience in the UVM FabLab, I see the students practicing the following skill sets mentioned in Forbes every day.

• Ability to work in a team structure
• Make decisions and solve problems
• Communicate verbally with people inside and outside an organization
• Ability to plan, organize and prioritize work

We’ll only know if we ask and track. If we can tell the educational makerspace story and back it up with solid metrics, we’ll improve the likelihood that our maker initiatives have staying power in the U.S. It’s also a way to start connecting the dots between makerspaces and flexible learning pathways (i.e. Vermont Act 77), active learning, hands-on learning and Project Based Learning.

Special thanks to the Malcom Cooke of ThinkBox at Case Western, Jennifer Buckley from the University of Delaware and J. Linsey of Georgia Tech. They were the presenters in Session 14: Metrics, Data and Measuring Impact and inspired the following top ten list.