LIS 505 - Software Evaluation Criteria
- Hardware compatibility.
A lot of this is the kind of thing
that you can check for on the box
(or in the basic requirements published on a vendor's Web site),
but there may be particular problems
that you have to find out about elsewhere.
Mostly, this amounts to an operating system question
(Windows, Mac, Linux?),
but the specific CPU may be sometimes be important
(a newer program may not run correctly on an older CPU,
or an old program may fail on a system with too high a clock speed).
Software will typically have a minimum memory requirement to run
and a recommended amount of memory to run efficiently.
Sometimes, certain features of the software will work
only if more memory is available.
- Graphics cards.
Software may not work correctly with certain graphics cards
(not just display incorrectly,
but also malfunction in other ways).
You may be able to avoid these problems
by changing the graphic acceleration settings
(possibly affecting the functioning of other software in the process)
or by updating the drivers.
The most common piece of information is
how much space the software takes up on the hard disk.
There may be different installation options
which require different amounts of hard disk space.
If the software comes on disk,
you also need to have the right kind of drive
to read the distribution disks.
some software is designed not to run
unless one of the distribution disks is present.
If the software is download in compressed form from a Web site,
it will quite often require additional space on the hard disk
for temporary files created during installation.
This is not much of a concern with Windows machines,
since software almost always uses the respective printer drivers.
You may want to check
that the software can accommodate the paper sizes
that you are going to use
and will avoid printing to unprintable areas.
- Other peripherals.
Many packages require the use of a pointing device
such as a mouse
(or a software alternative,
such as the Accessibility Options in Windows)
for at least some operations.
Some packages require an Internet connection,
either for registration
or to access additional resources
such as clip art.
Certain keyboard shortcuts may not work
with compact keyboards
like those on notebook computers.
Specialized packages may be designed to work with type-specific peripherals;
a speech generator with a sound card,
or a character recognition software with a scanner.
- Software compatibility
- Operating environment.
Apart from the general question of operating system family,
you may have to consider the specific version.
Some programs that run perfectly well under Windows 98
fail completely under Windows XP,
and vice versa.
There are occasional reports of software that ceases to work
after an operating system patch has been installed.
In addition, other installed programs such as virus checkers
may cause some software to malfunction.
Having the wrong version of a device driver installed
is another common source of woe.
- Other software required.
This may mean a runtime system,
such as that for Visual Basic applications;
or it may be that the software package being considered
is just a plug-in or add-in to some other software package.
Some software borrows DLLs from other packages
(for example, from Internet Explorer for displaying Web pages),
or requires a standard database management engine
to be running on a server.
- Software quality
Does the software do the kinds of things that you want it to do?
Are additional features also likely to be useful,
or will they just get in the way?
Does the software tend to crash or hang,
or crash the operating system?
Does it suffer from memory leaks
(not releasing memory when it is done with it,
so that the machine eventually ends up low on memory)?
Do its settings seem to change mysteriously for no reason?
- Correctness and adherence to standards.
Does the software do what it says it does?
Does it use interface elements such as text boxes in a standard way?
When it creates files,
do they adhere to the relevant standards,
or do other applications experience problems in reading them?
- Ease for all users.
Beginners should find it easy to get started using the software.
Experienced users should have plenty of shortcuts
to increase their efficiency.
There should not be any cliffs in the learning curve,
where learning the next thing suddenly becomes much harder
than learning the previous thing.
Are there different options available for installation?
Does the software adjust well to different screen dimensions,
color depths, and font sizes?
Can different interfaces be chosen to suit beginners
and more advanced users?
How many of the default settings can be modified
(for example, can you tell to program
to start browsing for files
somewhere else than in My Documents)?
- Use of keys, mouse, and menus.
Package functions should be accessible
both via the menus and via shortcut keys.
Menu items should show the associated shortcut key values,
so that users can find out easily what they are.
Menus should be well organized,
with clear captions.
Mouse actions should be standard
(left click to select or execute,
right click for a pop-up menu,
drag to move or resize).
Keys should be used in conventional ways
(Tab to move between fields,
F1 for help, etc.).
- Online help.
Text should be clear and use language correctly,
with appropriate headings and subheadings.
Unfamiliar terms should be defined and explaned.
Organization should be logical.
All information should be readily accessible for reference;
for longer help files,
this means having both a topical index
and a search function that works correctly.
Charts, diagrams, and screenshots should illustrate where needed.
What is said in the help files
should reflect exactly what is the case in the software itself.
Any links to other documentation
should be correct, complete, and useful.
- Training and documentation
Many of the same points apply to printed documentation
as are noted above for online help.
The availability of guides produced by third parties
should also be considered.
If the package includes tutorials,
consider how much they cover
and whether they provide real interactive learning
or simply encourage the user to click through a series of steps
without taking in what they are doing.
This may be provided by vendors
or by third parties
such as educational institutions.
Relatively simple software may not require much user training.
- Vendor relations.
You need to know what the local cost is to you or your organization;
this may mean taking account of taxes, shipping and handling,
currency exchange, and customs charges.
While less expensive software often has a fairly simple sticker price,
more expensive or specialized packages
may have more complex pricing systems;
price may depend on whether the buyer is a non-profit institution or not,
or, in the latter case, on something like annual sales revenue.
Competitive or upgrade prices
may also be available,
as may special rates for multiple licenses.
Some software licenses are not sold, but leased,
requiring ongoing expenditure
if you want to continue using the product.
- Copy protection.
Technical copy protection
(as against legal sanctions on unauthorized copying)
is found especially on games, educational software,
and expensive niche products.
One method is to require the presence of a physical object
(such as an installation disk,
or a "dongle" attached to one of the computer's ports)
in order for the software to run.
Another method is to require activation
after the software has been installed
in order to access all its features
or so that it will continue to run after a given date;
depending on the activation scheme,
this may mean that the software cannot
later be transferred to another machine
(or even run on the current machine if it is substantially upgraded)
without contacting the vendor
and perhaps paying an additional fee.
About all you usually get here is a limited time period
in which you can ask for a refund.
Software developers are notorious
for avoiding providing guarantees of the quality of their products.
There may be a limited period
during which you can download free updates
from an Internet site.
Various fees may be applicable
for more extended maintenance plans.
- New releases.
Consider how often new versions of the software are being released
and whether you will wish to upgrade at each new release,
at every second new release, etc.
Is there a special upgrade price?
Sometimes, uggrading to a new release is made free of charge
to those who bought an old version very recently.
- Assistance with problems.
This might be by telephone, fax, or e-mail,
or though an online user discussion board.
Larger vendors may provide a range of support options
from free for very bare-bones support
through various prices for different degrees of premium support,
with the highest levels covering, for example,
response every hour of every day.
Read what independent reviewers have to say about the product.
Be aware that reviewers for commercial magazines
may pull their punches to some extent
to avoid offending advertisers.
You may also see product announcements,
which are not real reviews,
but just repeat information supplied by the vendor.
Postings to discussion groups
may range from "astroturfing" by corporate shills
to malicious attacks;
look for discussions that seem to provide a good balance
of judicious comments.
For major software purchases,
such as library automation packages,
you should ask vendors to provide references,
whom you should contact independently.
For any software selection,
you can contact others who have some experience with the software
to get their reactions.
- Market share.
Market share is not always easy to determine,
though surveys reported on various Web sites
may be of assistance for certain kinds of software.
Market share is an important consideration for several reasons:
a product with a high market share
is likely to meet some minimum standard of quality
(though it is not necessarily the best product in its class);
if the product has a high market share,
the company is more likely to stay in business
and so continue to supply support and upgrades
(you can also check on the prospects of the company as a whole, of course);
even if the company goes out of business
or ceases to support the product,
there will be a large group of other users out there
who may be able to provide assistance;
file formats used are more likely to be supported by other products;
third-party documentation will be more plentiful and varied;
new and prospective employees
are more likely to be familiar with the software.
Last updated November 19, 2003.
This page maintained by
E-mail (text/plain only): firstname.lastname@example.org
Faculty of Information and
University of Western
Canada, N6A 5B7