Setting Standards for Evaluation

With almost two months experience as a Skype Journal Associate Editor, it has become apparent that a Skype Journal editor could become a test technician for every product with aspirations to work with Skype. However, time is a limitation and one needs to somehow define that gray border between evaluating a product and providing consulting on product management issues such as user interfaces, installation and configuration conflicts, Skype API integration and even Skype Certification. Rather than define a boundary here are the priorities I see in triaging our selection of products to evaluate:

  1. Is the product or service Skype Certified? If so, it has certainly passed one set of criteria that are becoming tighter as Skype develops experience with its partner program. Most of the items below are included in the Skype Certified criteria (SC) but I repeat them for emphasis.
  2. (SC) Does it include User Manuals, either in print or online? At least have them available for assisting the installation and operation. (Yes, I actually read them to assist with the objectivity of my assessment of a product when I run into issues.)
  3. (SC) Does it install seamlessly? Any installation or configuration precautions? Does the installation avoid conflicts with previously installed drivers and software? (On this one we recognize that devices with Windows Audio drivers will have to have the default Windows Audio device reset after installation — a Windows issue, not a product issue. On the other hand any installed software cannot wipe out currently installed drivers – as some have done.)
  4. (SC) Can it be installed and configured within 20 minutes? This is a bit subjective but if I can’t install the product within this time frame, the typical Skype User will not have the patience to complete an installation and configuration.
  5. Does it use the Skype client as the primary Skype Contact manager? Is one option for calling and answering one’s Skype Contacts simply the basic means of calling and answering available through the Skype client?
  6. (SC) Have the Skype API’s been used? While the API set is not complete (#1 request is for a Call Transfer API), they are relatively comprehensive and have facilitated the production of some excellent products.
  7. Does the product provide a definable User benefit or is it just “nice” technology? As a business unit within eBay, Skype is looking for third party applications, services and products that build both usage and its User base.

One overarching statement of Skype’s philosophy towards Certification::

Overall, we apply the same criteria to the solutions we certify [as] to Skype itself. Our basic feeling is that [since] “Skype is free Internet telephony that just works”, we expect the solutions we certify to behave in the same way. If a solution doesn’t work in a straightforward way, throws unclear errors upon install, and or requires a lot of backtracking to get it to work, we won’t pass it.

As we develop the Skype Journal Labs brand, this is the first of several guidelines that will be used to assist with setting priorities and providing some level of objectivity to our product reviews. Next:: key criteria in product evaluation.

About Jim Courtney

Bringing over thirty years' experience in the sales, marketing and management of cutting edge technology businesses.

No comments yet.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.