Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information
•Download as PPTX, PDF•
8 likes•13,827 views
This presentation explains how I use Neo4J as a database for a tool that calculate spreadsheet metrics.
Report
Share
Spreadsheets are graphs too: Using Neo4J as backend to store spreadsheet information
- 1. Spreadsheets are graphs too! Felienne Hermans (@felienne)
- 2. Spreadsheets are graphs too! Felienne Hermans (@felienne) In this slidedeck I explain how I used Neo4J to store information on spreadsheets
- 3. Ehm...spreadsheets? They are so tably? Are you sure they are fit for a graph database?
- 5. Spreadsheets are mislabeled People often think of spreadsheets as data, but...
- 7. Spreadsheets are code I have made it my life’s work to spread the happy word “Spreadsheets are code!”
- 8. Spreadsheets are code I have made it my life’s work to spread the happy word “Spreadsheets are code!” If you don’t immediately believe me, I have three reasons* * If you do believe me, skip the next 10 slides ;)
- 9. 1) Used for similar problems
- 10. This tool (for stock price computation) could have been built in any language. C, JavaScript, COBOL, or Excel. The problems Excel is used for are often (not always) similar to problems solved in different languages.
- 11. 2) Formulas are Turing complete
- 12. 2) Formulas are Turing complete I go to great lengths to make my point. To such great lengths that I built a Turing machine in Excel, using formulas only.
- 13. Here you see it in action. Every row is an consecutive step of the lint. This makes it, in addition to a proof that formulas are Turing complete, Also a nice visualization of a Turing machine.
- 14. 3) They suffer from the same problems
- 15. 3) They suffer from the same problems
- 16. 3) They suffer from the same problems
- 17. 3) They suffer from the same problems
- 18. In summary: both the activities, complexity and problems are the same
- 19. So if spreadsheets are code, can we apply software engineering methods?
- 21. In my dissertation, I defined smells for spreadsheet formulas
- 22. Turns out, Fowler’s code smells are easily transferable to spreadsheets
- 23. Pop quiz: what smell is this?
- 24. It is the ‘feature envy’ smell
- 25. See how easily this applies to spreadsheets
- 26. To analyze smells, we save spreadsheet info to a database
- 27. This is the data model that I am storing to the database. The basics are pretty simple.
- 28. This is the data model that I am storing to the database. The basics are pretty simple. But cells can refer to each other, either directly (i.e. =A7+A9) =A7+A9
- 29. =A7+A9=SUM(A1:A5) This is the data model that I am storing to the database. The basics are pretty simple. But cells can refer to each other, either directly [=A7+A9] or through a range [=SUM(A1:A5)]
- 30. This is the data model that I am storing to the database. The basics are pretty simple. But cells can refer to each other, either directly [=A7+A9] or through a range [=SUM(A1:A5)] In the case of a range, the range itself will points to the cells it contains. =SUM(A1:A5) A1..A5
- 31. You know the saying that if all you have is a hammer, everything is a nail to you. This is what happened to me. I did not think about what type of database to use.
- 32. SQL You know the saying that if all you have is a hammer, everything is a nail to you. This is what happened to me. I did not think about what type of database to use. I just started banging with the good ol’ SQL hammer I had been using for ever.
- 33. Number of worksheets in a spreadsheet Which started out just fine!
- 34. Number of cells in a spreadsheet Still pretty okay
- 35. Number of connected cells for a cell But, in order to calculate the ‘feature envy’ smell, we need the total number of connected cells. So both direct and through a range.
- 36. Number of connected cells for a cell But, in order to calculate the ‘feature envy’ smell, we need the total number of connected cells. So both direct and through a range. Let’s start with direct.
- 37. Number of connected cells for a cell
- 38. Number of connected cells for a cell But, in order to calculate the ‘feature envy’ smell, we need the total number of connected cells. So both direct and through a range. Let’s start with direct. Now look at the range part.
- 39. Number of connected cells for a cell
- 40. Number of connected cells for a cell
- 41. Number of connected cells for a cell Things start to get iffy when we combine these two query parts.
- 42. Number of connected cells for a cell
- 43. Number of connected cells for a cell Things start to get iffy when we combine these two query parts. Not only is the query quite big, also this happens.
- 44. Number of connected cells for a cell
- 45. If your tools reach their limits, this has to tell you something. So I started thinking.
- 46. Maybe this is not a nail…
- 48. Maybe I need a different tool
- 49. Maybe I need a different tool It was at this time that I attended a talk about Neo4J. And the strange thing is, I had seen a few talks about Neo before. But this time it ‘clicked’, because I was suffering from the problem that Neo could solve.
- 50. So I ended up with this model. Still spreadsheets, worksheets, cells and links.
- 51. So I ended up with this model. Still spreadsheets, worksheets, cells and links. But the ‘prec’ relation can now refer to either cells or ranges.
- 52. Turning this
- 53. Turning this into this.
- 54. Turning this into this. I wouldn’t say this is the power of Neo at work. It is the power of the right tool for the job. There are scenarios, for sure, where the situation is the other way around. But for my goal, Neo was a great fit.
- 55. Also, to be honest with you, I did not immediately write such super succint Cypher queries. My first attempt was something like this:
- 56. Also, to be honest with you, I did not immediately write such super succint Cypher queries. My first attempt was something like this
- 57. Also, to be honest with you, I did not immediately write such super succint Cypher queries. My first attempt was something like this This is basically a one on one translation from SQL to Neo. Still the two different ways of connecting. It took me a while to understand the power of traversal queries. Here’s another example:
- 58. Number of cells in a spreadsheet
- 59. Number of cells in a spreadsheet First Cypher attempt Still very SQLy
- 60. Number of cells in a spreadsheet Second (okay probably more like fifth) attempt. No more where, directly matching a graph pattern. The power of Cypher :)
- 61. That’s all folks. Spreadsheets are code
- 62. That’s all folks. Spreadsheets are code Don’t justhit things with the one hammer you know
- 63. That’s all folks. Spreadsheets are code Don’t justhit things with the one hammer you know Neo is cool for graph like structures
- 64. That’s all folks. Spreadsheets are code Don’t justhit things with the one hammer you know Neo is cool for graph like structures It makes queries easier
- 65. That’s all folks. Spreadsheets are code Don’t justhit things with the one hammer you know Neo is cool for graph like structures It makes queries easier But it takes some getting used to for SQL minded brains
- 66. Spreadsheets are graphs too! Felienne Hermans (@felienne) That’s all folks. Spreadsheets are code Don’t justhit things with the one hammer you know Neo is cool for graph like structures It makes queries easier But it takes some getting used to for SQL minded brains Liked this talk? Visit my site for more