This Stack

Decision Making and Computational Creativity

Posted on January 3, 2017January 3, 2017 by jonrgrover

When I think about where automated/computer decision making exists in this stack, it is in level 6 – computational creativity. You see, level 5 – artificial intelligence analyzes the world around it, and level 6 synthesizes things from its understanding. One of the things that can be synthesized is decisions. Level1&2:data. Level3&4:information. Level5&6:understanding. Level7&8:reproduction.

When building endemes by hand you are making decisions, so this form of building is a creative action. Endemes systemetize the construction of information, so endeme building can systemetize creativity.

Each Level Has its Own Native Structure

Posted on October 11, 2016 by jonrgrover

I will start with an example.

Here are examples at each level to handle a user-customer relationship:

Level 1 – RDB: A join table:
——————
user customers
Level 2 – OO: A few classes depending on use and need:
——————————————–
a user table with a customer objects list or a customer id’s list depending
a customer class
a user class
a ‘client’ data structure that contains both objects (references to them)
a user-customer relationship class
Level 3 – IOP: Three endeme sets defining user-customer fields:
—————————————————–
a UserIdentity set
a CustomerIdentity set
A relationship set
Level 4 – KR: Two nodes and a relationship edge:
————————————–
customer node
user node
relationship edge

Native Organization Schemes

Level 1 – RDB – relations
Level 2 – OO – hierarchies
Level 3 – IOP – matching/(or memberships?)
Level 4 – KR – graphs

For matching, each piece of data is part of whatever it matches well. For membership, each piece of data is part of multiple things.

The First Three Organization Schemes Are Implemented Through Four Types of Tables:

data tables – usually have a large number of rows focused on data
lookup tables – usually have a smaller amounts of rows focused on data [OO]
endeme tables – usually have a small number of rows focused on ‘type’ [IOP]
join tables – plumbing used to connect data tables to tables in a n to n relationship [RDB]

Given the needs of various part of a project, work with one or more of these for different purposes.

Resulting in These Structures:

hierarchy – oo is especially good with these [like lookup tables]
join – uniquely relational [join tables]
search – generally information oriented [endeme tables and endeme structures
data – everyone works with data to some extent
graphs – above the level of database tables?

Then adding two more types of tables:

Endeme implementation tables – endemes
Graph implementation tables – knowledge representation

Gellish?

These two new groups of tables implement endeme and knowledge representation structures (information structures) in a database. They generally set somewhat apart within the database in their own little engines.

Wisdom is a Computational Creativity Function

Posted on September 28, 2016September 28, 2016 by jonrgrover

Wisdom is a Computational Creativity Function. The computer has to create what it is going to do by applying its understanding gained through artificial intelligence.

An Example:

Data: (Raw) Red, 192.234.235.245, v2.0
Information: (Meaning) South facing traffic light on corner of Pitt and George Streets has turned red.
Knowledge: (Context) The traffic light I am driving towards has turned red.
Wisdom: (Applied) I better stop the car!

The Example Converted to Levels:

0. Raw Data
1. Stored Data
2. -> Meaning Data types and objects and hard defined structural interpretation
3. -> Meaning Information with structural (and conceptual?) context as meaning
4. -> Meaning knowledge representation with relational context to user
5. -> Context knowledge and what various outcomes and actions mean
6. -> Applied wisdom – stop the car!
Level 6 is computational creativity. The computer has to create an action.

Context and meaning are both built up through multiple levels rather than just being at one level each.

An Inner Platform is a Good Thing?

Posted on August 4, 2016August 10, 2016 by jonrgrover

In figuring out how to work with information inside a computer program, I am working toward building an inner platform. The thing I have to watch out for is an anti-pattern called the “Inner Platform Effect”. The inner platform effect often starts out with a generic database that gets added to and enhanced until it becomes a replica (usually a poor and incomplete one) of one of the systems that it is built on top of. Most often the kinds of inner platforms built are replications of relational databases and operating systems.

It is possible to build an inner platform that replicates the capabilities and organization of a relational database. It is possible to build an inner platform that replicates the capabilities and organization of an operating system. It is possible to build an inner platform that replicates the capabilities and organization of an object oriented software development system.

In data oriented software development the inner platform effect is an anti-pattern. The Inner Platform Effect – “The Inner-Platform Effect is a result of designing a system to be so customizable that it ends becoming a poor replica of the platform it was designed with.” This “customization” of this dynamic inner-platform becomes so complicated that only a programmer (and not the end user) is able to modify it.

Three things tend to lead to the creation of an inner platform

Users want something that they can modify themselves without having to have a programmer involved.
Programmers want to build something very generic because they do not know what precise needs their user have.
Programmers may realize deep down that the way forward in software development is to build a level above the level they are currently working. This may be the real reason programmers keep building them and suffering the consequences. Without information structures and theory, you simply can not build an inner platform that has added more capability than a database or object oriented approach. Any or all of these may be in play.
Programmers look at repetitive field handling code and think to themselves that there must be a way of abstracting it. You see (to oversimplify the situation) each column in a table ends up being a member in a class. A whole bunch of members tend to look the same, get set, changed and erased and displayed the same way, and the thinking is that there really should be a generic abstracted way to handle them.

Avoid the Inner Platform Effect

When you are building an inner platform there are some things you can do to avoid the inner platform effect:

Do something the underlying platform cannot do or can do very poorly. This is my goal – to build an inner platform that works with information as well as data.
Working with information is something that a database or an object oriented programming can barely do if at all. Usually when someone attempts to use one of these technologies for information, they end up with a snarl of excessively complex, nearly unmaintainable code.
– Endemes provide work-with-able information about the items in the ‘inner platform’. Information about the items is what inner platforms are normally missing.
For simple information: avoid objects and attributes in the endeme tables. have only endeme strings, labels and ids
- have the endeme id’s in the tables (objects) directly
- the endeme is the value
for endemized attributes/data:
avoid objects in the endeme tables, each endeme for the data is its attribute
– the endeme is the attribute, the endeme’s value is the data, the label is the name of the attribute
for endemeized objects: This would be a generic database and full inner platform.
I don’t know how to avoid building an inner platform here. I need much more experience working with 1,2,ad 3. above before I can hope to work with endemized
objects without suffering the inner platform effect. The trick is, that in order to get this experience, I need to build a bunch of inner platforms, have them die their expected gory death due to the inner platform effect, and eventually figure out how to build a true information programming platform that does not suffer this.
need to use level 4, can not do this properly until level 3 is well understood
this would build an actual inner platform.
– endeme lists would avoid the problem of not using the database for some of its excellent features.
this is because the database would need to support custom indexes, (since information has no order) and functions and triggers to power those indexes, (since the database does these very well)
Level 4 programming will be based on an inner platform. We need to build an inner platform for level 4 information programming. This will allow us to use knowledge representation in programming. Note to self: look to see if there are any successful knowledge representation inner platforms out there. OWL comes to mind.
AI programming will be based on the knowledge representation programming and is possible once an inner platform is built for level 4 programming. The inner model has a column for id, key/field, and value.

More thoughts on avoiding the inner platform effect

how do I avoid the inner platform effect?
- keep the inner platform really sparse in features
- do a hybrid system
- have columns for id(must have), field(in only its information format), and value(must have)
  - access by information, not ‘table’ and ‘key’
  - but an endeme list has endemes with labels, access by information, the endeme makes it different
  - have no search attributes, put these in regular table columns
- don’t let a user modify it, only for programmers
- build a solid infrastructure to hide the implementation details
- implement in really slow steps, taking years and learning how to do each step well before moving on to the next one
how do I use the RDB for what it’s good for?
the question is what can I do with information?

Building a Successful Inner Platform

The inner platform effect is an intuition by programmers that we need to build a level above the data levels. However without the information oriented programming theory to base it on, inner platforms just generally only implement things that already exist. Thus they usually fail to build a level above data levels.

I want to take the inner platform above both the database, the OS and the object oriented program level. The challenge is to have the inner platform reach out to the rel platform it replaces to use the performance facilities of the real platform below. The inability of an inner platform to do this is the most common downfall for the inner platform.

An inner platform for a database:

table
column
row
relation
pk
value

An inner platform for object orientation/xml/json:

class
member
object
type
collection

An inner platform for an operating system:

path
folder
filename
parent
child

Generic Data Models

– object attribute value

From C2: Definitely not an anti-pattern, and it can often greatly simplify table design for some corner scenarios. Apart from giving rapid prototyping benefits it also helps with handling ‘jagged’ data. Instead of introducing null columns using GenericDataModel enables us to save only relevant data thereby actually decreasing complexity. Reporting and manual labor against the table can be handled via views as mentioned above. not

Entity–attribute–value model

The entity: the item being described.
The attribute or parameter: typically implemented as a foreign key into a table of attribute definitions. The attribute definitions table might contain the following columns: an attribute ID, attribute name, description, data type, and columns assisting input validation, e.g., maximum string length and regular expression, set of permissible values, etc.
The value of the attribute.

From Wikipedia: As noted above, EAV modeling makes sense for categories of data, such as
– clinical findings, where attributes are numerous and sparse.
Where these conditions do not hold, standard relational modeling (i.e., one column per attribute) is preferable; using EAV does not mean abandoning common sense or principles of good relational design. In clinical record systems, the subschemas dealing with patient demographics and billing are typically modeled conventionally. (While most vendor database schemas are proprietary, VistA, the system used throughout the United States Department of Veterans Affairs (VA) medical system, known as the Veterans Health Administration (VHA),[1] is open-source and its schema is readily inspectable, though it uses a MUMPS database engine rather than a relational database.)

We Need an Inner Platform for Information

We need an inner platform because the db and the language do not provide what we need and we need a platform that will.
the problem is based on members

The DRY Principle and Information

Posted on July 26, 2016 by jonrgrover

DRY

In software development, DRY stands for Don’t Repeat Yourself. Conventionally this principle gets implemented by frameworks like ASP.NET MVC and Ruby on Rails by having a single source for column/field names, either the database or a class.

Framework Challenges

The challenge with these systems is to manage the sources. This requires high integration, and makes it very difficult for these frameworks to play with others. They are also very solidly based in a data oriented approach, meaning that they institutionalize hard coded context. With hard coded fields even when there is only one source for each, framework APIs become inflexible.

An Information Based DRY Framework

An information approach would treat each field as something to be processed and characterized rather than just as a field name and a type and other hard coded attributes. Information could provide flexible relationships between fields, flexibly created new fields, and control for the source of the field names, labels, types, and parsers.

Interacting with Another Framework

An endemized English vocabulary could be used to come up with an automatic signature for each of the columns in a database, each of the fields in entity framework, and each of the members of various data classes. These could be glommed. I have an endemized signature vocabulary for about 5000 words already available. Adding synonyms and two word definitions, and including another 1000 very common database column words could push this to 10000. Relationships could be identified from class memberships, operations and stored procedures. In this way an endemized framework could interact with MVC and Ruby on Rails without a lot of code written (once this framework was written).

An API to Anything

A further DRYing of code could happen by making the endemized framework a source. Once the framework above works, then this step becomes simple. And the endemized information framework can be a conduit to the other sources. We can build an API for interacting with the endemized framework giving other systems access to fields and rich metadata about fields in apps and databases. Endemizing provides flexible metadata, so there could also be an API to adapt the generalized framework to specific sources.

The Dreams of Computer Programming

Posted on July 18, 2016July 18, 2016 by jonrgrover

The dreams of computer programming:

The dream of writing a true AI program.
The dream of writing programs to meet our needs and desires and to provide benefits to us, for example including increasing productivity.
The dream of extracting information from data by writing a program.
The dream of a general purpose problem solver that for example can fly your car, cook your eggs, play chess with you and write a symphony etc. the key idea being that it is able to tackle whatever problem you might set it.

We have not met these dreams yet. Instead, true AI is still very far away. Only a fraction of our needs and desires get met. Instead of writing programs we use big honkin’ data mining and big data tools. We still write specific programs to do specific things.

I think the problem is that software development has reached a glass ceiling because we have not developed level 3 programming yet.

Level 3 programming requires three things

We need general programming, domain programming, and level 3 programming frameworks:

General programming is what we do right now with object oriented coding and object oriented code reuse.
Domain programming has to do with mixing our programming with concrete exececutable models of combinable, permutable concept lists focusing on the concepts and concept groups of the domains. i.e. endemes. This is just another definition for endeme sets.
A level 3 programming framework allows us to execute the executable models of combinable, permutable concepts. Rather than just complaining, I think it is important to also provide solutions to resolve what I complain about. Therefore, I am working on a level 3 framework. My latest framework is posted here.

What is Programming?

Posted on June 24, 2016 by jonrgrover

Programming is all about defining the meanings of words and using them to do things.

Our next step is to write programs that handle the meaning of words or something analagous to that. I could call this ‘Managing data using word meanings’. ‘Finishing the processing of the meaning of words’. or ‘Finishing the processing of words’.

I am thinking about the need to make a case for the existence of level 3. Very little work gets done on this level as far as I know. So maybe level 3 does not exist.

My Case for Level 3

My case for its existence is – there are bits, numbers, words, and sentences

Level 4 is all about relationships
level 3 is all about meaning
level 2 is all about data
level 1 is all about storage

level1 1 2 and 3 all seem to have something to do with the meanings of words:
level 1 provides the storage
level 2 provides the processing of words using hard coded context, for its precision.
level 3 completes the meaning of words
level 4 puts the words in context

Maybe Typing is the Problem

maybe compilers are not the problem. maybe typing is the problem. Not just strong typing, any typing. We need something more flexible, and yet still processable. Xml is a form of context, however it has the hard coded typing problem. Why is hard coded typing a problem? Because hard coded context itself can not be processed. Is that the real problem?

The idea of meaning for data and context to manage it

Context is critical. relationship is critical but it is not primary. The meaning of a word by itself is critical. We need to learn how to handle words. We have made words hard processable. Now we need a layer on to op the hard processable words that finishes
the job of lower level programming.

What will it take to finish the job of meaning aside from relationship?

1. context processing – contexts are a sort of relationship but what I mean by context is context by class and table.
– the ability to process that context is important – is this level 3 or level 4?
– the part that is level 3 is the part that manages the contexts of levels 1 and 2
! we need a context manager.
2. endematic meaning – the meaning of the word (mostly) by itself – in other words the definition of a node in a graph.
3. the meainig of words
the endeme set to be applied – is this level 4?
4. a system to store a words meanings and endeme sets

endeme – enumerated meme – this is the definition of an endeme in A.I. terms.

Every Diagram Has its Use, Here is an Overview of Software Development Diagrams

Posted on June 22, 2016June 22, 2016 by jonrgrover

I have gone and looked at the various software development diagrams available on Wikipedia. I have focused on the graphs (nodes and connections). The natural level for graphs is level 4 (knowledge representation and ontologies). However level 4 has a broader scope than just (K.R. and O.). Here is what I was able to suss out.

Level 4 – High Level Information Programming – Knowledge Representation

Level 4 proper diagrams mostly have to do with relationships.

Anchor modeling
Concept map
Conceptual graph
Conceptual model
Concept-oriented model
Context diagrams
Data flow diagram – this diagram has a long and storied history, As level 4 develops, I expect to see this diagram rise in importance again.
Domain model
EXPRESS-G
Information flow diagram
Information model
Object model
Object-role modeling
Semantic network
Semantic Web
Top Level Ontology

Some level 4 proper diagrams are based more on hierarchies than relationships. Of course a hierarchy is a kind of relationship.

Argument map
Cladistics
Document Object Model

Level 4 for Other levels

Here are some level 4 diagrams that apply to other levels:

4(5) Fuzzy cognitive map

4(3) Nets within Nets
4(3) Specification and Description Language
4(3) Composite Structure Diagram (UML)

4(2) Abstract syntax tree
4(2) Class diagram (UML)
4(2) LePUS3
4(2) Tree structure

4(1) Entity–relationship model
4(1) Hierarchical database model
4(1) Network model

Diagrams for Other Levels

Here are diagrams that apply directly to other levels, although one could
certainly build systems at level 4 that would process these.

Level 1 – Data Storage and Relational Databases

Level 1 isn’t really very diagrammatic. It mostly relies on grids. The diagrams that apply best seem to be level 4 diagrams that manage level 1 such as entity relationship diagrams.

Jackson structured programming
Symbol table

Level 2 – Data and Object Oriented Programming

Many of these diagrams have to do with various sorts of process flow.

Algebraic Petri net
Behavior tree
Business process mapping
Business process model
Control flow diagram
Decision tree
Directed acyclic graph
DRAKON diagram
Dynamic model
Enhanced Transition Schematic
Event-driven process chain
Finite-state machine
Flowchart
Flow process chart
Function model
Interaction overview diagram (UML)
Petri net
Sequence diagram (UML)
State diagram (UML)
State Machine
Use Case Diagram (UML)

Level 3 – Low Level Information Programming

I have had a lot of trouble trying to find any diagrams that apply to level 3.
These are the closest, but they are very thin gruel.

Activity diagram (UML)
Composite structure diagram
Business logic
Component diagram (UML)

Level 5 – Artificial Intelligence

Knowledge representation is a substrate for artificial intelligence. I expected to find more of these. Perhaps as level 4 develops there will be more in the future.

C-K theory
Extended Enterprise Modeling Language

Level 6 – Computational Creativity

Computational Creativity is a severely underdeveloped level. I didn’t expect to find anything, but here is one that may make sense.

Mind map

Extracting Information Value from Maintenance and Legacy Code

Posted on May 31, 2016 by jonrgrover

We can extract information value from maintenance and legacy code. Legacy code contains lots of business rules. It also contains lots of level 3 design information.

Level 3 Design Information

Bit fields, Boolean fields, types, status variable, and enumerations all contain design information that can be collected into endemes. For example, int Microsoft Great Plains, the AF40110 table contains a whole series of flags that can be combined into an endeme set, as does the CM00002 table,

The CPO40001 company table contains type and flag information related to approvals and variances. The FA00100 table contains property and asset type information. The FA00200 table contains depreciation types, indicators and flags. The FA45000 table is filled with all sorts of types which could be combined into one or more endeme sets related to retirement, transactions and year start and end status and types. And so on. IN Microsoft GP I have identified about 3000 columns that could be combined into and managed as information using endemes.

Each endeme set thus extracted is a piece of information design which can ten be ported to new programs during application migration efforts. This is because each endeme set aggregates a group of concepts that operate together in combination.

As a Maintenance Programmer

This holds true with each database and application I have worked with as a maintenance programmer. One of the challenges of maintenance programming is that you can not break the functionality of the applications because the application is usually in production and real users are using it and real companies rely on it. In order to refactor legacy programs you need a serious reason to do so. Information oriented software development provides that reason.

Legacy programs exist at level 1 and level 2. Refactoring them to extract and exploit level 3 allows them to be much easier to maintain in the future, allows them to be much easier to extend in the future, allows them to be much easier to migrate in the future, and allow them to be much easier to test in the present as changes are made. In addition, refactoring portions of them to level three will extent their life.

Businesses Face These 11 Key Relationship Challenges

Posted on May 19, 2016May 19, 2016 by jonrgrover

Here are the 11 key relationships that businesses must handle properly to succeed.

Relationship with technology — technology, adaptability, design, innovation.
Relationship between quality and growth — balancing quality and growth.
Relationship with people — customers, employees, vendors, clients, development.
Relationship with with self — organization, integrity, internal metrics, vision.
Relationship with with talent — managerial & employee talent, hiring, training, compensation.
Relationship with with cash flow — profit, revenue, health insurance etc.
Relationship with with capital — capital, investing, investors.
Relationship with with data — information <-> data <-> internet.
Relationship with with the market — new markets, competitors.
Relationship with with the future — planning, risk, uncertainty.
Relationship with with society — regulation, social media, reputation.

These 11 relationships come from the Business challenge endeme set below

ABCDEFGHIJKLMNOPQRSTUV|lt Label            complements    num   descriptive
----------------------+--------------------------------------------------------------------------
A]C-E-            -T- |A. Adaptability/tech    D tech->u      2   Technology/technial infrastructure/adaptability/embracing change
[B]     I      P R    |B. Bottom line          H money->u     1    Increasing profit and revenue/margin erosion
 [C]       -M-  -R-   |C. Customer             V u->cust       3  relationships/loyalty/marketing/service/insight/latent needs - Business managers must properly identify the needs of their consumers or business customers/Customer experience/clients
  [D]                 |D. Design/innovation    A u->tech    0     analagous to energy vs acceleration
 -D[E]                |E. Exploding data       J data->u     1    data explosion/data to informtion conversion
 -C-[F]               |F. Funding              M invest->u          capital
     [G]              |G. Growth               Q G/Q?         1    Business growth/scaling/globalization/multiple versions
      [H]             |H. Healthcare           B u->money     1    insurance coverage
       [I]    -P-     |I. Integrity            O metrics->u       3  Internal metrics/integrity/performance/speed vs quality/metrics and integrity/trust
                      |J. J-?                  E u->data    0     information to data conversion/artificial creativity/representational interfaces/reports/dashboards/internal reflections on the internet
 -C-     [K]M-        |K. Know Market          N mrkt->u       3  knowledge/the market/the competition/protecting market share
          [L]M-       |L. Leadership           T ldr+wkr      2   management
 -C--F-    [M]        |M. Manage finances      M u->invest       3  Financial management/cash flow
AB-       -M[N]       |N. New business         K u->mrkt       3  Attract business/marketing strategy/adjacent markets/time to market
             [O]      |O. Organization         I u->metrics    0     structure/incentives/organism/ideas/vision/mandate/governance/internal churn/mining IP
              [P]R-   |P. Problem solving      U event->u      2   and risk management
-B-  -G-       [Q]    |Q. Quality              G Q/G?         1    balancing quality and growth
                [R]   |R. Regulation           S old/new?       3  and compliance/government
-B-           -PR[S]  |S. Social reputation    R new/old?      2   reputation/social media/publicity/Brand/marketers with analytical skills and digital media savvy
 -C--F-H-     -PRS[T] |T. Talent/Staff         L wkr+ldr       3  Talent/hiring/recruiting/competencies/people
              -P--S[U]|U. Uncertainty          P u->action      2   uncertainty/Planning/Strategy/long term/short term/tool box approach
   -E-              [V|V. Vendors & employees  C vend->u     1    career path for techies/advancement/training/learning/Vendor, Employee and Supplier Engagement/Vendors
----------------------+--------------------------------------------------------------------------
ABCDEFGHIJKLMNOPQRSTUV|

                                             +- A. Adaptability/technology
                              +- technology -+- D. Design/innovation
             +- x-ware -------+                      |
             |                |  growth/     +- G. Growth
             |                +- quality ----+- Q. Quality
             |                                       |
inward ------+                               +- I. Integrity
             |                +- self -------+- O. Organization
             |                |                      |
             |                |              +- L. Leadership
             +- human --------+- talent -----+- T. Talent/Staff
                              |                      |
                              |              +- V. Vendors/employees
                              +- people -----+- C. Customer
                                                     |
                                             +- N. New business
                              +- the market -+- K. Know Market
             +- the industry -+                      |
             |                |              +- E. Exploding data
             |                +- data -------+- J. J-?
outward -----+                                       |
             |                               +- S. Social reputation
             |                +- society ----+- R. Regulation
             +- the world ----+                      |
                              |              +- U. Uncertainty/planning
                              +- the future -+- P. Problem solving
                                                     |
                                             +- B. Bottom line
                              +- BH ---------+- H. Healthcare
money --------- BHFM ---------+                      |
                              |              +- F. Funding
                              +- capital ----+- M. Manage finances

A. Adaptabilty/tech challenge endeme set:

ABCDEFGHIJKLMNOPQRSTUV|
----------------------+
B E I R |. LEGACY TECH Legacy technology - endematic extration of information entities
R T |. RIGHT TECH correct technology to apply to somethng - endematic choosing of technology
I S |. SCALING scaling and infrastructure - ? - IT no IN
E T |. EMBRACE CHANGE training and embracing change - endeme created ad hoc teams to advocate and teach
I M |. INTEGRATION integrating systems - endematic data integration through data semantics
F N Q |. NEW QUESTIONS new features/questions/dashboards - endematic glomming of databases
F P |. PROCESSES support of new/changed processes/flows - ?
A F R |. DEV FOCUS technology development focus selection - endematic resource allocation
L N |. NEW/LG BALANCE balancing resources: legacy vs new - endematic ROI and resource allocation
U |. USEABILITY making sure people can use it - endematic proxies and wrappers
D M V|. MIGRATION migration from old to new systems - endematic data version integration through data semantics
----------------------+
ABCDEFGHIJKLMNOPQRSTUV|

I. Internal metrics challenge endeme set:

—————————————–
qualitative software development metrics?

J. J-? challenge endeme set:

—————————-

information to data conversion
artificial creativity
representational interfaces
reports
dashboards
internal reflections on the internet
data production for search engines

K. Know market challenge endeme set:

————————————

|. adaptability to markets – endematic niche focus, and market direction resource focus

O. Organization challenge endeme set:

————————————-

|. software and technology re-use       – endematic software repository & can endemes make software easier to reuse? – endematic source code analysis
|. document and knowledge re-use       – endematic document repository
structure
|. incentives, clear ‘policies’       – endeme set: keep job/make no mistakes, take risks/investigate opportunities, cost savings, personal incentives, budget/project based, structure/department based
organism
ideas
vision/mandate
governance
|. teamwork, team mutual knowledge       – endemes to show each team member about the others, and which tasks to assigne each
|. fluid organization           – endematic ad hoc team building

P. Problem solving

Spot the key problem

T. Talent/Staff challenge endeme set:

————————————-

(hiring/assignment/teamwork)

talent/aptitude
competencies/skills
experience
corporate culture fit
domains
temperaments

We Can Build Mutual Languages for Talking with Users

Posted on May 10, 2016 by jonrgrover

Programmers could build mutual languages for talking with users. One of the difficulties of programming is figuring out what the users are asking for. Programmers base their understanding and therefore their terminology on Level 1 and Level 2. Users seem to operate at Levels 4, 5, and 6 (Knowledge, intelligence, and creation). When we develop Level 3 programs and designs we develop mutual languages between users and programmers at the same time.

Technology Suicide Sickens Me

Posted on May 5, 2016May 6, 2016 by jonrgrover

To see people committing technological suicide, and to rope me in as being part of it, leads me to feel nausea. I feel like throwing up when I have to build reports. Reports are a form of technological suicide, reports are where data goes to die! Instead of being able to use the information extracted in the report, it goes onto a piece of inaccessible paper or goes into an inaccessible format. yock.

Instead of using the information that reports create to build this stack upward, the information just sits there. My focus is to build this stack, and to have to know that my energy is going to push technology in the opposite direction offends me. There I said it. It is a vile and unconscionable waste of my customers’ mind space. It leads to them thinking that they can not work with the information once provided, using programs that can be built to meet their needs. It leads to deadening of the soul and the senses. Also the business rules are essentially buried in the database. Reports are bad.

I go back and forth between wanting to cry and wanting to vomit.

Programmers Should Stop Misleading Users

Posted on May 4, 2016 by jonrgrover

Users see the wrong things so they can’t ask for the right things. Programmers work really hard to convert data to information to show the users based on user needs. These are ad hoc conversions of data to information as data is moved to the UI for users to use. But then the users don’t know how it has been produced and ask for things in the ad hoc information terms the programmers were able to shape things into which are the wrong terms to use with a programmer because the programmer does not live in the world of information.

This is not as much a problem in new development, but this is a huge problem in maintenance development where usually different programmers are doing the development than those who did the original development. This may be part of why maintenance development is so expensive.

Programmers mislead the users by creating a world for them that is not organized the same way and does not reflect what is behind the UI.

My goal is to build a common information world that both programmers and users can use.
Then they can interact more easily, productively and without misunderstanding. Level 3 and Level 4 could be used for this purpose.

We Can Build New Report Generation Tools

Posted on May 3, 2016May 3, 2016 by jonrgrover

We have report generation tools like Crystal reports and SSRS. We could do better that these. I propose building a tool called ‘fReports’ which stands for free reports. It would have the following qualities gained implemented in the following levels:

Level 4 to layout report.
Level 3 to make it dynamic and flexible.
Level 2 to show data.
Level 1 to store the data.

Searches would include implementation in levels 1 and 3. Of course I have to get Level 3 and 4 working first 😉

The Flip Side of Removing Technical Debt is Adding Information Value

Posted on April 27, 2016July 1, 2016 by jonrgrover

When managing software development, there needs to be an amalgamation phase to convert types and bits into endemes. (- once things are stable). Somewhere between code rewriting, and code refactoring, is code information amalgamation which is adding information value.

The flip side of removing technical debt is adding information value. You can go through programs to remove technical debt. This removes problems. You can also go through programs to extract information structures. This adds value.

Here are some things that can be extracted from code, some to remove problems, some to add value which I found through the Google search “Extracting * from code”:

extracting code from code bases [adds value or removes technical debt]
extracting a method from code [adds value]
extracting a model from code [adds information value]
extracting the application architecture from code [adds value]
extracting the data from code contents [adds value or removes debt]
extracting the model from a code [adds information value]
extracting the strings from code [removes technical debt]
extracting this data from code [adds value or removes technical debt]
extracting abstractions from code [adds information value]
extracting action words from source code [adds information value]
extracting Algorithms From Code [adds value]
extracting Annotations from Code [adds value]
extracting API examples from test code [adds value]
extracting Authors from Source Code [adds value]
extracting automata from code [adds value]
extracting candidate fingerprints from code [adds value]
extracting code blocks from code [adds value]
extracting code metrics from code [adds information value]
extracting Colors from Code [adds value or removes technical debt]
extracting configuration from code [adds value]
extracting content from code [adds value or removes technical debt]
extracting data from code [adds value or removes technical debt]
extracting Definitions from Code [adds information value]
extracting dependencies from code [removes technical debt]
extracting descriptions from code [adds information value]
extracting design from code [adds information value]
extracting documentation from code [adds information value]
extracting dynamic behavior from source code [adds value]
extracting existing context from code [adds information value]
extracting Facts from Perl Code [adds information value]
extracting Gold from Legacy Code [adds value]
extracting high-level designs from code [adds value]
extracting Hybrid Automata from Control Code [adds value]
extracting information from code [adds information value]
extracting IR tokens from code [adds value]
extracting JPF Options from Code [adds value]
extracting kernels from code [adds value]
extracting knowledge from code [adds information value]
extracting meaning from code [adds information value]
extracting meaningful metrics from source code [adds information value]
extracting measurements from code [adds information value]
extracting Messages from Code [adds value]
extracting metadata from code [adds value]
extracting methods from code [adds value]
extracting models from code [adds information value]
extracting multi-threaded capabilities from code [adds value or rmvs debt]
extracting other relationships from code [adds information value]
extracting parallelism from legacy code [removes technical debt]
extracting protocol state machine from code [adds value]
extracting questions from code [adds information value]
extracting relationships from code [adds information value]
extracting rules from code [adds information value]
extracting secret credentials from your code [removes technical debt]
extracting software architecture from code [adds value]
extracting source content from code [adds information value]
extracting specifications from code [adds information value]
extracting specs. from code [adds information value]
extracting Speedup From C-Code [adds value]
extracting SQL from the code [adds value or removes technical debt]
extracting SQL statements from application code [adds value or rmvs debt]
extracting state machines from code [adds information value]
extracting strings from code [removes technical debt]
extracting summary keywords from code [adds information value]
extracting type signatures from code [adds value]
extracting UML constructs from code [adds value]
extracting unnecessary knowledge from code [removes technical debt]
extracting Values From Code [adds value or removes technical debt]

I’m a simple guy

Posted on April 14, 2016April 14, 2016 by jonrgrover

I’m just a simple guy, which makes me wonder why I’m out here doing research and invention. Perhaps God delights in having the simple do important things. When I visit Programmers Stack Exchange I realize that lots of people know a lot more than I do. So why haven’t they pushed programming to the next level?

Sometimes We Need to Embed Meaning in a Computer Program

Posted on March 10, 2016April 11, 2016 by jonrgrover

Information can be thought of as data that has meaning. Digging around the web I found some thing that people seem to think are involved in meaning. Data can be supplied with meaning through:
– description – data semantics
– importance – endemes (seriousness)
– relationship – knowledge representation (causal)
– process
– community
– use-purpose
– security – access/permissions
– communication
– situation/setting/relevance
– logic/propositional logic/truth value
– function
– history/time/date
– event/noun/verb/adjective
– implementation
– commonness
– statistics
– grammar semantics/Chomskyan internalist semantics
– variables system
– text before and after it – “Words and sentences that occur before or after a word or sentence and imbue it with a particular meaning.”
– creator

These are mostly forms of context. The “theory of meaning” is discussed at http://plato.stanford.edu/entries/meaning/. Interesting keywords are meaning, context, importance, description, relationship, purpose.

Components of Meaning Endeme Set

Here is an endeme set regarding some implementable components of meaning.

ABCDEFGHIJKLMNOPQRSTUV|ltr Label        Levels: Implementation Tech    additional terms/description
----------------------+--------------------------------------------------------------------------------------
A]            (P)-S-  |A.  Access       ..3.... Permissions            security, role based security
[B]C-  -I--L-         |B.  Bridges      .2..... SOAP etc.              Communication, interaction, channels, lines, dataflow
 [C]                  |C.  Community    ......7 Agents                 Communities of agent - Level 7.0?
  [D]            -S-  |D. *Description  ..3.... Data Semantics         Semantics
   [E]              -V|E.  Events       ....5..                        <Event/verb>
 -C-[F]            -U-|F.  Frequency    1......                        Commonness,Ubiquity
     [G]  -L-    -S-  |G.  Grammar      ....5.. Syntax                 language
  -D- [H]     (O) -T- |H.  History       <ugh>                         past,temporal/time/date/timestream/timeline/change
       [I]    -P-     |I. *Importance   ..3.... Endemes                Priority - weighting - Endemes - Level 3.0
    -F- [J]        -U-|J.  Job          ....5..                        Function(,Use)
         [K]    -R-   |K. *Kowl. repr.  ...4... Knowledge representatn Relationship - Level 4.1 (or 4.0, 3.9)
          [L] -P- -T-V|L.  Logic        ....5.. Propositional logic    truth value, truth
-BC-   -I- [M]        |M.  Making       .....6. CBR                    Implementation,built,construction,internals
 -C-        [N]O-     |N.  Noun/object  .2..... Objects                <Noun>/category/object
 -C-         [O]      |O.  Origin       ......7                        Creation,Creator
    (F)       [P]     |P.  Process      ...4... Workflow               flow
A-         -M- [Q]    |Q.  Quality      .....6.                        Adjective,Modifier/Decorator
                [R]S- |R.  Relevance    ....5..                        Situation/setting
           -M-   [S]  |S.  Statistics   1...... Statistical measures   median/mean/mode/spread
                  [T] |T.  Textual pos  ....5..                        textual position, text before and after it
              -P-  [U]|U.  Use          .....6.                        Purpose, roles
     -G-   -M-    -U[V|V.  Variables    .2..... Variable system        gaming/toy/universe model description
----------------------+--------------------------------------------------------------------------------------
ABCDEFGHIJKLMNOPQRSTUV|                 2332632 - 2 or 3 at each level except A.I. where meaning is the focus

Levels are: 1)R.DB., 2)O.O., 3)D.S., 4)K.R., 5)A.I., 6)C.C., 7)C.o.A.

Tips and Tricks for New Programmers

Posted on March 10, 2016 by jonrgrover

Build a data structure optimized for how you know you are going to use it.

how is it keyed, how are you going to access it?
if you don’t know the best way to design it, start with a sub-optimal solution until you write more code.
sub-optimal often means a group of lists or dictionaries of primitive types which operate in synch. Then you will eventually see that you need one two or more classes to keep the primitives in and refactor optimally based on your needs.

Build things at least twice, preferably three times

the first time to understand the requirements.
the second time to figure out the best design.
the third time to do the implementation.

If you build things in a generic way so that they can be unit tested, they some of the things you build will be reusable in the second and third iterations.

Draw ERD’s to understand the database

An entity relationship diagram is a diagram showing how tables in a relational database relate. The boxes are the tables and the line between them are the relations. ERDs are like having a map to learn how to get around a city.

use crows feet for the relations.
ignore types but include each column name, key name and table name.
make the table boxes 2 inches (or 5 cm) wide.

Draw ‘calling’ trees to understand a large code base

They are like having a map to learn how to get around a city.

draw it so that the callers are on the left and the called methods are on the right.
use arrows to indicate non-direct calls (like event invocations, web service calls etc.).
the step through debugger is your friend in this.

Estimate a project at 4 x (new db columns + new db tables + information characteristics + relevant target fields) hours

I just did a project with 22 columns in 4 tables, 12 characteristics, and 22 target fields in about 260 hours.
the phases were 1) prototypes and requirements(10days), 2) design document(1day), 3) convert to production quality code (5days), 4) finish production code(10days), 5) convert to using web services(4days), 6) testing(1day), 7) deployment(1day) ~= 260 hours.
make the estimate 5 x instead of 4 x for a kind of program you have not written before.
If you do not get the project done within the estimated time, bump that factor of 4x or 5x up to what it needs to be for your experience level for succeeding projects.

Useful items to have in documents are:

WHY. why the customer need this and how they will use it
REQS. requirements.
OODD. object oriented design.
INFO. information analysis and design.
UISD. User interface layouts and story boards.
RDB. relational database design.
NETS. networking and integration needs.

Depending on the size of the project these could be in one or multiple documents. Here is how these items relate:

  OODD--REQS
  /  \  /  \
RDB--INFO--WHY
  \  /  \  / 
  NETW--UISB

Different Talents are Stressed for Data and Information Oriented Programming

Posted on February 22, 2016April 11, 2016 by jonrgrover

A somewhat different talent endeme is useful for a data oriented software developer and an information oriented developer.

DOSD:

Inductive reasoning
Practicality
Deductive reasoning
Creativity
Q-recognition
Reason
Advocacy

IOSD:

Q-recognition
Deductive reasoning
Creativity
Reason
Practicality
Inductive Reasoning
Advocacy

Deductive reasoning and creativity are important for both.

Hard Coding and Soft Coding

Posted on February 16, 2016February 19, 2016 by jonrgrover

Soft coding is introduced on a level by level basis, with each level benefiting from the soft coding at its own level and the ones below it.

level -2 (paper and pencil) hard codes document version
level -1 (notepad) soft codes document version and hard codes literal order
level 0 (spreadsheets) soft codes literal order and hard codes literals
level 1 (rdb) soft codes literals (as variables) and hard codes instances
level 2 (oo) soft codes instances and hard codes variable context (as members with hard coded names)
level 3 (data semantics) soft-codes variable context and hard codes highly complex data structures
level 4 (knowledge representation) soft codes highly complex data structures and hard codes program functions
level 5 (artificial intelligence) soft codes program functions and hard codes synthetic operations
level 6 (computational creativity) soft codes synthetic operations and hard codes program instances
level 7 soft codes program instances as agents and hard codes communities of agents
level 8 soft codes communities of agents

These are general tendencies, you can find exceptions.

	republic on Freetext endeme set
	Charl on I Miss Wiki Wiki Web
	Wilhelmina on Staffing a Full Stack Pro…
	Bette on Creativity HCI
	Diandra on Permutations are Information