Forcing new google gadget to load, not from cache

One common challenge when working with Google Gadgets is forcing a new development version to load, not reusing a cached version. A quick search on finds several suggestions and conversations about how to do this, but they are not complete. Renaming the gadget for each save is overkill and not worth it. The developer gadget does not appear to correctly skip the cache in Safari. A little playing reminded me that the key is to edit the bogus querystring parameter to make the URL unique each time you re-add the gadget.

So, as the first suggestion above says, add a parameter like ?nocache=1 to the end of your gadget moduleurl when adding it.

AND... when you need to make another edit, change the parameter! The 2nd time you need for force a reload, set the url to ?nocache=2

Next time, ?nocache=3

And so on...

Google Gadgets

I started experimenting with Google Gadgets today - and I am very impressed with how easy the iGoogle and OpenSocial frameworks are to use. This is a great example of how a good framework, combined with some on-demand hosting services and mashups of REST-based data sources can make lightweight (or "rowboat") application development very simple. Mixing data from multiple sources via easy-to-integrate URL's is a great development paradigm for creating operational dashboards or quick glance BI reports.

In my particular case, I was looking to be able to quickly glance at a page of stock charts to see whether any stocks I'm currently following are at an interesting point. FinViz.com is a wonderful freemium site for simple analysis, but to follow 4 or 5 stocks required just too many searches and clicks. But FinViz follows the good Web 2.0 pattern of making their charts publishable and linkable through a URL, so you can combine them to make a dashboard or mashup very easily. (More proof that REST-based web services are better for distributing your data or enabling partners to easily integrate. SOAP is ok for heavy weight system integration, but if your goal is to get your data used, use REST!)

A few hours of reading the docs and scripting and now I've got a configurable stock dashboard integrated with my iGoogle home page. I just have to decide how to handle the charts when the gadget is not maximized -- any suggestions? :-)

Integrating Content in Customer Workflow Applications

As I discussed in my last post, it is critical for information firms to distribute and deliver their content where their customers want to purchase, read, and use it. In addition to needing to support more platforms and mobile devices, publishers need to integrate their content in to their customer's workflow applications. Typical customers do not purchase data, reports, articles, or analyses for fun - they purchase these content pieces to help them accomplish their larger business objectives. This might mean checking the credit history of a loan applicant, scouring scientific literature for a grant application, or analyzing the legal precedents of an upcoming case. In each situation, the goal of the customer is not to get a document to display on her bookshelf, but to get access to the information in the course of performing a larger task.

Obviously, the key to make this easy is to not force the customer to leave their workflow tools and go to a discrete content website but to INTEGRATE the content in to the workflow tool. In many cases Information Providers are moving up the value chain and selling those workflow enabling tools - for example:

• LexisNexis Litigation Workflow Tools
• D&B Risk Management Tools
• Thomson Reuters Healthcare Clinical Evidence Solutions

In each case, the Information Provider adds value to the raw underlying content by providing a business process focused (or Knowledge Worker focused) workflow tool to help the knowledge worker turn the content from "information" in to "knowledge". Of course, this value-add is in addition to the direct content value-add through aggregation, classification, entity recognition & linking, and analytics applied in the Information Factory.

By building these Workflow solutions on top of standard REST or WebServices API's, the provider is able to leverage the existing content repositories, search, and enrichment capabilities without having to create duplicate product stacks. And, as I mentioned last time, the same content could be directly integrated with a customer's proprietary workflow tools or data. A well thought out and designed API enables many additional distribution channels and revenue generation options with a high ROI by enabling both provider built workflow tools and customer integration. And of course those same API's can support the new mobile applications that customers want.

There is also a very fast growing and productive middle ground for Information Providers to integrate with - Microsoft Sharepoint. This is one of the fastest growing products in Microsoft history, and used (admittedly to varying extents) by many, or even most, Corporate customers. The ability to provide pre-built Web Parts that customers can easily install in to their Sharepoint portal to integrate a provider's search and document retrieval capabilities are very powerful and provide a low-cost, very easy way to move beyond a generic internet content web site to a more integrated Enterprise application. Providing additional Sharepoint workflow enabled components can enable an information provider to provide the full range of on-premise, Enterprise capability without the cost and complexity of Enterprise application development and maintenance.

Sell Where Your Customers Buy

Many marketing or business planning discussions eventually hit on the seemingly self-evident notion that you should "Sell What People Are Buying." (For a humorous discussion of this, see this blog) In addition to this basic truth, it is also important to "Sell Where Your Customers Buy." Think of the sidewalk entrepreneurs in New York City selling ice cold sodas or bottled water out of a cooler on the corner in lower Manhattan on a hot summer day. These vendors are able to charge a premium for a cold drink because they are right where the people are walking and thirsty - they are selling Where their customers want to buy.

For Information Providers, this truth is becoming very important. Not only is the Information industry moving heavily from a print to an all-digital delivery model, and not only are the roles of intermediaries and centralized procurement "locations" (such as corporate libraries) being diminished, and not only do Information Consumers have more choices and more access mechanisms than ever before, but in addition the easy access to information is changing the notion of information gathering from a stand alone task to simply one small (and hopefully automated) step within a larger process. Let me explain what I mean in a little more detail.

We all know that Print media is turning Digital. See the news stories about declining Newspaper readership, Amazon's (and BN's, and Border's, and...) push for digital books, the frequent discussion about the "Google Effect" on content providers, and the Open Access Initiative. The digital content is available on many platforms - the web, your mobile phone, smartphones, tablet computers, and e-readers. Information Consumers expect to be able to read the articles, analyze the data, and do their jobs on any of these devices, anytime during the day, on the train, in the airport, or wherever they are. This is what I mean by "Sell WHERE Your Customers Buy" - you need to be able to sell the article, or provide the data visualization, or the drill-down analysis on any device connected to the internet. If your valued, long-term Customer is sitting in the airport, curious about something she is not going to wait to get back to her office to use her subscription service to order a new report - she wants to download it to her phone, right now. If your service does not provide that, she will go to your competitor's app and get her Information right now. Your valued, long-term, trusted relationship goes right out the window because your Web 1.0 Information Delivery website cannot meet her expectation of instant information access.

The effect of this is to require every Information Provider to need an iPhone app. Well an iPhone app and an Android app. Oh, and an iPad app. And probably a Galaxy and Playbook app. And of course, a mobile friendly version of the web site. You can see, this quickly adds up.

Early attempts to create mobile apps usually involved creating a separate application stack, content collection, or Information Delivery Channel. This was the quickest, and least expensive approach to get the first mobile app out and does work. But the cost curve is not pretty, and you soon need to maintain and support 10 different applications, delivery channels, etc at 10x the cost for only marginal revenue increases.

The better approach is to build an API. Not just a dirty hack of a url that can be used to access content, but an actual, thought-out Service Oriented API that provides billing, metering, search, retrieval, and easy integration for apps. Now those 10 apps are just thin layers reusing the core SOA enterprise services and can be built and maintained cheaply. Better yet - if you have a loyal community who truly values your content, they will wind up building innovative apps on every platform you have never heard of to access and embed your content. This is because your customers are empowered to scratch their own itch, and build an application to solve their own personal or particular corporate need. And it turns out that a lot of times your customers know how to use your content better than your own product development team -- so the apps they build and the embedded data usage they put together are more useful to your customers than the pre-built website or stand alone applications you might design. Your customers become your distributors and open up entire new retail channels you were not even aware of.

So... you get more sales channels, at a cheaper delivery cost, with more customer value from enabling your content through an API. Sounds like a Win-Win-Win!

The New CMOS - Cloud Mobile Open-source Social

CMOS is a well-known acronym in chip manufacturing meaning "Complementary Metal-Oxide-Semiconductor". According to Wikipedia:

"CMOS" refers to both a particular style of digital circuitry design, and the family of processes used to implement that circuitry on integrated circuits (chips). CMOS circuitry dissipates less power than logic families with resistive loads. Since this advantage has increased and grown more important, CMOS processes and variants have come to dominate, thus the vast majority of modern integrated circuit manufacturing is on CMOS processes.

In other words - because this way of designing and building chips has distinct advantages in improving efficiency and enabling new features, it has become the dominant approach to building chips. Any mainstream (non-experimental) chip manufacturer has settled on this process because of the inherent benefits.

There is a new CMOS taking over the software development industry - "Cloud Mobile Open-source Social." New applications, whether they are commercial software products, internal enterprise applications, or B2C web applications (think "Web 2.0", Linked Data and related) are - by default - leveraging the combination of Cloud, Mobile, Open-source, and Social technologies to enable rich feature, scalable, always on applications that are developed faster and cheaper.

Users expect 24/7 access. Consumers expect to be able to share and tag and link content. Readers expect to be able to access articles and applications on any computer, any device, anywhere. This is a fundamentally different mindset than "traditional" application development and traditional publishing models. It requires new ways of architecting applications and new ways of thinking about data and content. It's a heck of a good place to work!

Ten Steps to Agile Software Development Process Improvement

In a previous blog entry, I mentioned 10 Steps to Improve Software Development Process. If you read these and pause for a minute, you'll notice that many of these are actually principles of Agile development. I want to expand on a few of these thoughts here.

1. Focus on the top 20% of features:
This is one of the primary drivers of value in Agile custom application developmentpractices. By prioritizing and rank-ordering every item in the feature request backlogs, only the most important ones are developed. By focusing on these Top 20%, you can often satisfy 80% of what end users want, and they can start using the system sooner, to add to the profitability of your company. (If the most important 20% of the features do not add to your company profitability, you should probably cancel the project!)

2. Break things up into smaller projects:
Big projects turn in to huge projects. And miss deadlines. And run over budget. Reading the Standish Group CHAOS report figures on failed IT projects always makes me wonder why more people don't follow the simple advice to "not bite off more than you can chew"?

Again, the Agile concept of smaller, more frequent releases echoes through this item. Getting a working system in to the hands of users is always a good thing - that's why you are implementing the system to begin with! Giving an important and useful portion of functionality early is even better. This has many organization benefits - from psychological ones like proving that the overall program can work and the enthusiasm from a successful launch, to risk mitigation benefits like the ability to redirect spending on an investment after the first release if priorities change, to the practical one that it is a lot easier to notice a project in trouble if a specific release is over budget or late than if fuzzy 'milestones' are being missed.

5. Obtain user feedback
During the implementation process, keep the end users constantly in the picture. Show them early versions of the system - for instance at each Sprint Demonstration or at least with small, frequent releases. Let them give you feedback, and above all, let them change their minds without being punished. Trust your end users; they know what works and what does not - and they are the ones who are going to use the system every day! When they see working features, they may be able to better prioritize other changes or features to be able to complete an important business process or simplify many steps.

7. Mistakes are a way of learning
Remove the blame culture. Let people make mistakes quickly - failing early is much better than missing a delivery date. These so-called mistakes are part of the overall discovery process and will help lead and evolve to the eventual solution. If you blame people for mistakes (picking the wrong feature for a Sprint, not seeing a bug, very bad color choices) they will react to the blame and change their own behavior. Rather than being an active participant in making the project a success, they will become "followers", just doing what they are told - no way to get blamed there - and punching the clock. The good people will hate the blame culture enough to look for work somewhere better.

10. Something has to give in the Iron Pyramid (Quality, Time, Cost, Features)
The old Iron Triangle, which I've always thought should be an Iron Pyramid with Quality as an explicit dimension, still holds true today. In an agile development process you try to "bake in" high Quality by using unit tests, refactoring, engaged people, and frequent review processes. You fix the Time or at least the time cycles - every two weeks you have a Sprint release; and the Cost is essentially fixed based on the size and members of the team. The Features dimension is what gives - and that is where the prioritization comes in. By putting the most important features first, you complete as many in each Sprint as you can and know that you are achieving the most important features at any given time.

Over a longer time horizon, say a Release of 5 to 6 Sprints, you can adjust the Time and Cost dimensions, by letting the process run for more Sprints or deciding that enough Features are ready to stop this project.

I focus on the Iron Pyramid because we all know the truth. If you push people hard enough, they will relax some of the quality checks, and they will get one extra feature in. But you have lowered the quality level - maybe not enough to notice today, but you will pay for it in the future. Whether it is the performance testing that is skipped, leading to a slow web application, or "smelly code" that costs more and more to maintain over time you pay for the lapse in quality.

Data Visualization and Web 2.0

I love data visualization techniques. From my early days as a operations data analyst and all of my software development career, finding patterns in data and finding an easy way to convey the pattern through a graph or other visualization has always been fun. Working on custom application development projects that provided a picture of how the business was doing, where customers were spending, etc is fun. Now working with our Business Information Services clients to help create innovative approaches to information discovery and data analysis is fun. It really is true that often "a picture is worth 1,000 words."

I vividly recall a stubborn memory leak my team had been trying to track down for several weeks. This was a long time ago, in the days of VB6 COM dll's running inside ASP web pages, and we were pretty sure our code was not leaking. The team had found memory leaks before, and tracked every single one of them down to circular references in our COM object model so that the automatic release never occurred. Historically, it had been easy to find a leak by running a simple load script, executing each page thousands of time in isolation and watch to see which page shows the memory leak. But not this time. We had run the load tests several times and never found the leak. We scanned the code thoroughly. We added as many "set obj = nothing" safety lines as we could. But still the production web servers kept leaking memory, and we were forced to move the automatic restarts of the servers from weekly to daily and hope our band-aid would hold.

One day, I had some down time and decided to see if I could find a correlation between the memory used and what pages were invoked on the production system. An hour or two later, I had pulled all the IIS log files, gotten dumps of memory traces from the systems team and started my analysis. A bit of awk, grep, Access, and other quick and dirty processes later to pull out the data I wanted, adjust for timezones, aggregate hits to cumulative 15 minute buckets and otherwise line up the datasets and I was ready to plot the data.

Instantly, the answer of where the leak was was obvious. The two lines, cumulative hits to a particular URL and memory in use were nearly on top of each other. The correlation jumped out, completely overwhelming the noise of other URL's, pages, etc. This is the power of a good visualization. (Of course, it turned out that the leak was coming from a web services API proxy URL, not a page in the website that everyone had focused on! Since the proxy was not 'in' the website it had been ignored for weeks as the team hunted for the answer.)

Recently, some colleagues and i were discussing what areas Alliance Global Services provides solutions to clients in. This is a pretty broad topic, and we talked about the types of industries we serve (including our focus on Business Information Services), the geographies we serve (mostly North East US, from about Virginia to Boston), the types of services we provide (Custom Software Development,Application Architecture Analysis). And we talked about the easiest way to visualize our coverage areas.

Well, today I had a little downtime before the holidays. So I took a list of our client locations used some simple geocoding tools, and put together two quick samples of mapping in the Web 2.0 world - one using a Yahoo! map through batchgeocode.com and the other using the Google visualization API.

Batchgeocode.com made it very easy to process the first set of data and create a map but then you were stopped. Google was a different story - getting the map running required coding, but then I had full control. To see the first map, visit this blog on Alliance Global Services.

Obviously it's not perfect, but lots of fun for a quick afternoon's work!