by jsalem under Inside QuickBase

Over the weekend, we had to quickly undo a mistake we made. We broke some usages of the "rdr" (redirect) option for QuickBase API calls. We fixed the problem as soon as we were able to reproduce it. 

"What’s an rdr?", you ask. It’s a cool developer feature typically used when QuickBase provides backend database services for a customer’s application. It’s documented along with the other API features on the Developer Info Page. After going to that page, click on  "API Reference Documentation".

I thought you might enjoy a "behind the scenes" peek at this example of our process for rolling out upgrades and fixing critical bugs. It’s fairly representative of how we fix any significant user-visible problems.

First, mea culpa. While multiple people and automated tests cross-check our software before it goes into production, the bug was in an area of the product I’m responsible for and it should never have happened. Fortunately, in these cases the QuickBase team focuses on fixing problems fast and learning the lessons that will prevent them from recurring.

BACKGROUND

Back in October and November, we upgraded to a new version of our web hosting platform. This version provides more reliability and scalability. Unfortunately, the vendor introduced a new limitation which prevents us from accurately logging performance. Although customers don’t directly see this data, performance logging is very important to our system health monitoring and capacity planning. After working with the vendor to resolve the issue, we decided to change the interface technology we use. In addition to fixing the performance log problem, the new technology shows a slight speed improvement and should be significantly more scalable.

The new technology was implemented in our code and tested and deployed in our pre-production environment at the beginning of January. The change was not expected to have any impact on customers, so the testing was limited primarily to our automated test suites and ad hoc testing done while examining other features. Unfortunately, the testing for the "rdr" feature was not included in our standard automated test suite which focuses on the most common user tasks.

We deployed the change on 1/3 of our web servers last Wednesday night. The release went smoothly (so we thought) and there were no reported problems. Given its success, we made the change to the rest of the web servers last Friday night. Even though we expected no downtime, we still like to install upgrades during off-peak hours (generally, nightime in the U.S., and more specifically late Friday night). This gives us a chance to hear of any bugs before the heavy usage starts on Monday.

SOMETHING’S NOT WORKING

We first received a report of an unexplained API problem at about 10:45pm on Saturday. I reviewed the issue, however, on the surface, it seemed unrelated to the upgrade.  Unfortunately, there was not enough information in the report to reproduce the problem so I requested additional detail. We received a second problem report from a different customer at 5:30pm Sunday, this time with enough information to reproduce the issue.

To correct the problem, we rolled back to the earlier version of our software by 6:30pm — problem solved.

On Monday and Tuesday, we corrected the issue in our software and did some thorough testing of the "rdr" feature. Going forward we will test this feature as part of our automated test suite. Last night (Tuesday), we redeployed the new interface and so far everything looks good.

OBSERVATIONS

I want to highlight a few things about our process.  First, we do a lot of testing before new software goes into production. We know customers rely on us 24×7 and work hard to insure upgrades do not change an end-user’s experience of a QuickBase application.

Second, we time our releases to have the least impact on customers. Even if we expect no customer impact, we still make software changes during off hours to leave time for problems to be discovered, reported, and fixed before our peak hours.

Third, we react quickly to problems. Our support team monitors QuickBase 24 hours a day and are not hestitant to call any team member at any hour to resolve serious issues. Most problems with broad impact are resolved within minutes or a few hours. More commonly, for bugs that have simple workarounds or only affect new product features we will typically wait for a minor patch release (every few weeks), or a major release (every few months).

Fourth, we build contingency plans into all our releases. We have a reliable "roll-back" strategy which lets us revert to back to an earler version if a catastrophic problem occurs.

HELP US TO HELP YOU
If you notice something suddenly changes with your application, let us know by opening a support case!

Only rarely can we fix something if we can’t reproduce it. So make sure to give us enough information. This includes data such as:

1. The URL of the page that produced the incorrect data and information on how you got there.
2. The date and time you noticed a change in the application’s behavior.
3. The browser type and version you are using.
4. Any recent changes you made to the application.

The easier you make it for us to reproduce the problem, the faster we can react. In some cases, we may ask you to grant us access to the application. Our support team has no ability to look at your application unless you specifically grant them access.

While most reported issues are not software bugs requiring immediate patches, they still often highlight areas of the product that are difficult or confusing and that we need to take a look improving in a future release.

MOVING FORWARD
Even though this problem affected few users, it is an example of a type of issue that we take very seriously. We know our customers expect us to be available at any time and we are committed to meeting or exceeding your expections. Therefore, we react extremely quickly to any issues that affect data integrity or security, or those that have a big impact on the end user.

Wishing you all clear sailing with your QuickBase applications.

by rmcdonald under Uncategorized

        "How can I view or print a receipt for my QuickBase Subscription?"

        "What does this error message mean?"

Do you ever have a question about QuickBase but don’t know where to find the answer? To get you the information you want, we’ve enhanced our Web site support pages.  Now you can get fast answers to your most common questions and issues.

These answers and more are now just one click away. See for yourself! To check out the new pages, click here or select Help > Support from My QuickBase. 

by rmcdonald under Uncategorized

Last night, QuickBase elves were busy applying another small update to the QuickBase service.  Okay…they weren’t elves, but they did leave a couple of presents for you.  In addition to the usual small fixes, we reintroduced one feature and added a new one.

Users are typically deactivated to remove a user’s access to applications in an account, but sometimes, just like the cat, they come back. Therefore, we have reintroduced the ability for an Account manager to reactivate a user.  In order to deactivate or reactivate a user account the Account manager must own the e-mail address of the user.  To find out the criteria for determining whether you own a user’s e-mail address and more about reactivating an user account, please go to the Reactivating a User Account topic in the QuickBase Help.

Ever want to make a change to a field, but think twice about because you are not sure what impact it will have on existing views, forms, etc.?  Now, before you modify or delete a field, you can easily see where it is being used in the application.  On the Fields and Tables page, we have added a Usage button to the right of the field name.  Clicking this button provides a list of views, forms, fields, roles, emails and exact forms where the field is currently be used.  Clicking on the elements name in the list automatically takes you to the proper location to view or modify the field reference. For more information on checking field usage refer to the QuickBase Help topic, Checking Field Usage.

by Peter Fearey under Inside QuickBase

With QuickBase, we cover the spectrum from folks feeling it is expensive to others finding it cheap. Let me cover the cheap first, as these folks are delighted with the value they are getting from QuickBase,though sometimes they do worry… "How can you provide this service for this little $$?"

The answer is volume, and it is one that Intuit understands well. When QuickBooks first came out, plenty of business accounting packages were available for a few thousand dollars; QuickBooks was about two hundred dollars. There were people that would have spent the thousands for business software, who got QuickBooks for "cheap". However, there were many, many more that never would have purchased the several thousand dollar solution, but did purchase QuickBooks. So, when folks tell us that they’ve:

• avoided the $75,000 of a traditional CRM system;
• saved $50,000 in software, hardware, and training costs for project management software; and
• replaced a $200,000 event management system;

…all with QuickBase (and actual customer stories), we don’t worry that we are an order of magnitude less than the prices they would have spent. We want to provide this value to the masses of folks who could never have spent this type of money to get a high-quality solution like this.

Now, let’s cover the "expensive" side of the spectrum. On this one, I’m going to stay where we get the most questions, but if you have others you want answered, just comment and I’m happy to do what I can to explain. The most "it’s too expensive" comments come around data and file attachment space charges, as they are much more expensive than just buying your own disk space. This is an understandable comparison, so let me explain why we aren’t as crazy as it appears on the surface.

The charges for data and file attachment space for your applications aren’t related to the amount we pay for the storage, but rather are related to the value you get for your specific application. The comparisons to pricing on storage space don’t map, as those options do not give you the functionality around the data or file attachments that QuickBase offers. For example, with QuickBase file attachments integrate into your process, are searchable across attachments, and contain revision history and locking capabilities, etc. None of this comes with the disk drive you can buy.

The reason we allow pricing along these lines is that everyone is different. Some folks have lots of users, and little data space, but lots of file attachments, where as others have lots of users and lots of data space, with no attachments. Our objective was to give you flexibility.

The next place the "expensive" conversation comes is when folks are only using QuickBase with a small number of folks (<5) for only one application. This is a bigger subject that we can talk about more too. The one answer the kind of sums this one up is a quote from Todd Porter of Honeywell… "QuickBase is limited only by your own imagination." Rev up your imagination and get some more things going in QuickBase!

As always, we’d like to hear your thoughts on this. Pricing is a BIG topic and I’ve tried to be brief, so I’m sure there is plenty I’ve missed.