Entity Framework’s Change Tracking is Jenga for ORM’s

0

I recently spent an entire evening tracking down a crazy Entity Framework error:

"The operation failed: The relationship could not be changed because one or more of the foreign-key properties is non-nullable. When a change is made to a relationship, the related foreign-key property is set to a null value. If the foreign-key does not support null values, a new relationship must be defined, the foreign-key property must be assigned another non-null value, or the unrelated object must be deleted."

Oh now that’s a classic Entity Framework error. The problem is that the operation I was performing had no changes to the relationships. I was just setting some data on the object and saving it back. Everything else on the site worked, I could delete, add and update DB objects everywhere, it was just this one page, but why? So began a 6 hour crown and coke fueled debugging session where I had to “git reset” 4 times.

jenga

Entity Framework’s change tracking system is very complicated and far reaching. Operations that you perform on change tracked objects can have a big impact in different areas. MSDN has a good area on Change Tracking for POCO objects to get an overview. And there is a good answer on Stack Overflow on it as well.

I eventually tracked down the offending code, but the kicker, it has been working in the system for over 6 months! So why now all of a sudden did I get this error message? I would say that change tracking is a house of cards and that can be true, one issue or one mistake and it can all fall down, but really Entity Framework and it’s Change Tracking is a game of Jenga, something you did early can cause a catastrophic failure latter on.

Lets look at the offending code, that was almost completely separate from where I was getting the Entity Framework exception above:

public List<DepartmentGroup> GetAllGroupsForDepartment(int departmentId)
{
    List<DepartmentGroup> departmentGroups = GetAllGroupsForDepartmentUnlimited(departmentId);

    // Remove all Disabled or Hidden Users
    foreach (var dg in departmentGroups)
    {
         dg.Members = (from m in dg.Members
                       where !_departmentsService.IsUserDisabled(m.UserId) && !_departmentsService.IsUserHidden(m.UserId)
                       select m).ToList();
    }

    return departmentGroups;
}

The code above get’s all the DepartmentGroup’s for a Department, pretty simple. In Resgrid you can hide or disable users, so they won’t appear in parts of the system. So we remove them from the list of members and continue. This is where the issue lies, because we are removing Members, which was populated by Entity Framework and all change tracked it assumed that we wanted to delete the members that didn’t make back into the Members list. Leaving their DepartmentGroup reference null and causing the error when we tried to submit changes.

This from my experiences with Resgrid which is a cloud service company, SaaS deployed on Microsoft Azure, providing logistics and management tools to first responder organizations like volunteer fire, career fire, EMS, search and rescue, public safety, disaster relief organizations, etc. It was founded in late 2012 by myself and Jason Jarrett (staxmanade).

Like I said before this code has been in the system for 6 months without issue. But why did it break now and how did that code affect a completely unrelated part of the system? Normally in the app, either via the API or the web site, calls like this live in isolation and the context is scoped to the current request, giving us a nice Unit of Work Pattern. Every time that method got called it’s affects on the context were short lived and isolated.

We recently added a new feature to the Edit Profile pages where you could set a person’s group, so to get the departments groups we called that method, in the POST of the controller we re-get them to re-populate the view model. This is where we ran into the issue, change tracking detected the removal and when we tried to update our unrelated object the error was thrown.

What I wish is that Entity Framework would allow us to figure our what SubmitChanges() was trying to do, this would have made the debugging process far easier. Instead it was hours upon hours of work and was only found when I started commenting stuff out and removing it.

So if you are getting the error above and your not removing child objects from the database take a look at other operations your performing in the scope (this works well in your using a Unit of Work pattern). As a practice don’t remove objects from a change tracked collection unless you want to remove the object from the db, instead remove change tracking or spin the result into a new collection.

Forcing yourself to launch your product

So you’ve spent all this time building a product or service, when do you finally launch it for money? This can be one of the more difficult decisions that you have to make. As a developer we are hard wired to be passionate about our craft, especially when it’s our baby. Even the most detached developer gets that glimmer in the eye when it’s about the code they wrote for that critical process. When it’s your own product or service and a direct representation of you sending it out into the wild, and asking money for it, prepare for an internal tug of war like you cannot imagine.

rocket

Besides naming your company or product one of the most important emotional decisions that you will have to make is when to launch your startups product and asking people to use it and more importantly pay for it.

Some history, this is from my experiences with Resgrid which is a cloud service company, SaaS deployed on Microsoft Azure, providing logistics and management tools to first responder organizations like volunteer fire, career fire, EMS, search and rescue, public safety, disaster relief organizations, etc. It was founded in 2012 by myself and Jason Jarrett (staxmanade) and we launched it as a paid product in 2014.

The Resgrid SaaS runs on Microsoft’s Azure and was up there at the end of June 2012. We started charging for the service January 2014. That’s a year and half we had the product up there for free. Even with a year and a half of development and work, asking people to pay for it was a very difficult decision. There was more to add, more to fix, more to test, more to improve. it never ended.

So how can you force yourself to launch your product?

1.) Have a partner/s or a sounding board

Having someone with knowledge of the product or service, with experience in launching or good baseline knowledge I think is important to forcing you to think critically about your baby. An different perspective is incredibly valuable when internally your conflicted about a decision. Even if you don’t want partners in your endeavor then find someone who will listen and give advise. What you don’t want is someone who will blindly agree with you.

2.) Have a MVP spec and stick to it

At some point early in the process determine what your minimum viable product is, what would be the baseline someone would pay for? It can be one feature, or a set of features. But your not talking full on double rainbow, just the bare minimum feature set. Your MVP is important, it’s the baseline of what people will use, find useful and possibly pay for. But it gets it into their hands early. Resgrid launched on 6/22/2012 with a MVP (allow first responder personnel to denote if they are responding or not). Our mobile apps were dead simple:

Actions_Screen

Oh yea, now that’s an MVP. A lot of what we’ve added to Resgrid over the two years has been feedback from our customers, or as Jeff Atwood would say, Complaint Driven Development. This is something that I pride myself on and as a company Resgrid prides itself on, quickly responding to our customers feedback. But if we didn’t start with an MVP and get it into the hands of our users, we most likely would be a completely different product.

3.) Get rid of friction

Before you launch, get your development and deployment stories sorted. Think of this as softening the beach before the invasion. This gets rid of the excuse of “it’s too time consuming or costly to get it out there” (especially for SaaS products). At Resgrid we use Microsoft’s Azure coupled with BizSpark this lowered our cost and allowed us to iterate very quickly.

Remove all the friction that prevents you from focusing on your product. Use cloud providers, like Azure, to deploy your product to for getting it in front of your customers, even if you eventually expect to house it yourself in house.

4.) It’s good enough

Launching will never be easy, it will never go flawlessly, there will always be problems and people that hate it. That’s just the way it is. We iterated on Resgrid for a year and a half, pushing weekly updates and getting feedback. But it was still a difficult decision. There are still parts of the app that need more QA, more work, help and insight for the users, documentation, etc. But those elements and issues will always be there.

You will never polish every edge case, every combination or usage pattern. Instead of tying to fix all of those, focus on your “Happy Path” and ensure that works well. Then respond to customer feedback and fix the edge cases you didn’t think of or didn’t test.

5.) Set a date and stick to it, no matter what

I’m not a fan of hard dates for delivering features. I feel that this can cause a business to be less nimble in response to issues, customer requests or changing market conditions. But for your launch you need to just set a date and make sure the product is MVP complete about two months before that. Spending the remaining time fine tuning, testing, fixing critical bugs and preparing your marketing collateral.

Don’t neglect your public presence like blogs, twitter, Facebook, public site, etc. Unless you have a built in audience or are well known it can be hard getting the first user, even if the MVP is a free alpha. The goal with your launch is to get users on it as quickly as possible, get feedback and iterate as fast as you can.

Don’t worry about complaints, issues or other negatives

This is going to be tough, and I didn’t put a number on it because it’s not a ‘thing’ to try and get you to launch. No launch goes smooth, there will be issues and probably a lot of your early adopters will probably leave and never come back. Don’t dwell on the negatives, you need to push on and keep iterating.

Launching something is a rollercoaster, from major emotional highs (getting signups, paying customers and compliments) to some serious lows (crashes, unsatisfied customers and no signups). As the adage goes, it’s a marathon not a sprint. Don’t fear the negatives, learn from them and move on.

Your pricing model is more than your price

When creating a product or service there’s a point in time, usually pretty early in the process, when you start to form your pricing model. If you haven’t gone through it before it’s something that you very easily can get wrong. The first thing to get right is the price and the value proposition but that’s not what were going to talk about today, today it’s everything else and the presentation of your model.

pricing

Before we dive in here’s a little background. This is from my experiences with Resgrid which is a cloud service company providing logistics and management tools to first responder organizations like volunteer fire, career fire, EMS, search and rescue, public safety, disaster relief organizations, etc. It was founded in 2012 by myself and Jason Jarrett (staxmanade).

So my experiences are more related to SaaS subscription pricing models and starting from the ground up, making plenty mistakes along the way. Jason and myself are both experienced developers, but this is our first real foray into starting a complete company around a service.

When we started developing Resgrid we decided on a price we believed would have a great value proposition and provide a public service (proving a free plan to small first responder organizations). Next we started designing our tiers and limiters for each tier. This is where it’s easy to add confusion and thus paralyze your potential buyers into inaction.

Here’s a scenario; you’re a volunteer fire department with 15 Personnel, 2 Stations and 2 Units. Which of the following Resgrid plans is for you? Can you figure it out at just a glance?

Price1Price2

Don’t feel bad, I built the system and it still takes me a solid second to decode the leftmost option. When we launched our paid service at the beginning of the year and the feedback started trickling in. First, WTF are roles, and then what are groups. We heard those questions a lot. If your having to explain your pricing model, either the model is failing or the presentation is.

For Resgrid the Roles just wasn’t a good limiter, it’s was confusing and fuzzy when a department would need to push that and get an upgrade. Your limiters needs to be easy to understand and have well known and easy to identify trigger points; “I hired a new person”, “I added a new station”, “I got a new Unit” are all good, easy to understand limitations on the tiers. If it’s some arbitrary or fuzzy value, then chances are it was just added to try and more reasons to upgrade and not as a real value on a tier.

The other change we made was to rename Groups to Stations. We all know that naming and terminology is important, but when someone is using it to make a purchasing decision it’s vital. The names and terminology your using should be, or easily equate to, what your customers use. We weren’t doing that with Groups, as their primary use is to map to stations but the name was our term, not a standard industry term and thus had to explain it to our potential customers.

Below is my list for creating a pricing model, I’m sure it will evolve over time, but take these into considerations when designing your own.

  1. Get Your Price and Value Proposition right
  2. Keep your tier limiters small (no more then 5)
  3. Present them well (side by side/easy comparison)
  4. Limiters equate easily to customer trigger points
  5. Terminology used equates to what your customers use

Good luck on creating your pricing model, and if you feel it’s not working for you don’t be afraid to change and alter it. The worst thing you could do is leave a pricing model that driving potential customers away in place.

Go to Top