User-Organised Resources
Introduction
Users frequently struggle to access information or resources relevant to their needs, not because the information is scarce, but rather few services are available to help them find it. The fundamental aim of a digital library is to assist users access materials effectively and readily online with a variety of disorganised resources. This is not an easy task and is why few digital libraries are effective in doing so.
Digital libraries, especially the NDLTD, may improve research efficiency by implementing a set of enhanced user services to help users navigate the content. The goal of this research was to investigate whether this is true.
Research Objectives
Several academic search engines, the most well-known of which is Google Scholar, include capabilities to assist scholars in finding relevant content. However, their utility and efficiency, like most digital libraries, leave a lot to be desired. When compared to prominent academic search engines, Global ETD Search has even less capability.
This system provides users with the functionality to organise content via the use of reading lists. Reading lists offer users a way of managing their research in any way they choose. Users can create public or private reading lists containing any ETD available on NDLTD. Users may choose to create private reading lists that are only visible to themselves or a public reading list, which can then be viewed and accessed by all system users. Users can toggle the reading list between public and private with ease. These features aim to give users the ability to organise their research into different categories and preserve it for later reference. The final feature included in the system enables users to bookmark ETDs to a "read later" list quickly and easily. Bookmarking ETDs intends to be a quick and easy way to save ETDs of interest. To test the effectiveness of these features, the following research questions were be answered:
- Will public reading lists provide a useful recommendation feature to help users find relevant content?
- Will private reading lists provide a better user experience than the current bookmarking tools users have at their disosal?
System Development
The system was designed as a web-based application. The system design followed a layered architecture, allowing the development and refactoring of each layer to occur independently. The layers follow a Model-View-Controller (MVC) design pattern, meaning different layers perform the application’s Model, View and Controller functions. Figure 1 illustrates the architecture. The system makes use of the existing Global ETD Search features and data. All the ETD metadata existing on NDLTD is indexed and searched using an Apache Solr Index.
Model
The Model handled the storage and organisation of user data. The database was designed and developed in MySQL following the 3rd normal form. The database stores users, reading lists, reading list items and bookmarks.
Architecture of the System
Controller
The Controller handles the manipulation of content by the user when they use all the available user services. The system was developed as a Java web application. The Controller consists of Java Servlets that are used to manipulate the database with various create, read, update and delete (CRUD) methods and pass dynamic data to the View of the application. There are several Java Servlets that are responsible for the user profile, reading lists and bookmarking functionality.
The Controller is also responsible for handling the search queries performed by the user. When a user enters a search query, a Java Servlet receives a HTTP GET request from the View and passes the search query to the running Apache Solr index already mentioned, containing all the ETD metadata.
Homepage of the System
Performing a Search
View
The development of the View had a strict focus on user experience. The View presents all the user services to the user and allows the user to manipulate the system. Since the system is web-based, the View was developed in HTML, CSS and JavaScript to create a responsive and sleek interface. The design and development of the View had the aim of being simple and easy to use.
The system provides users with all functionality that already exists on Global ETD Search. Users can open the system and make full use of the search functionality just as they can on the Global ETD Search. However, if users decide to create a profile or log in to their already existing profile, they will have access to the set of enhanced user services offered by this system.
To see the system in action,
click here.
Performing a basic search on the system. Basic searches can be performed without the need of logging into a profile.
Creating a user profile. If a user would like to make use of the enhanced user-services, a profile needs to be created. Users are required to state their preffered username and their email address.
Viewing a reccomended reading list. After performing a search query and viewing an ETD, reading lists from different users are reccomended to the current user. If the current ETD being viewed is present on any other users' reading list, that list is reccomended to the curremt user.
Adding an ETD to a reading list. If the user would like to add an ETD to a list of their own, they click the add button and can either add it to an existing list, or create a new list to add it to. If a new list is created, the user can decide on the name of their new list, and then add the ETD to that list.
Bookmarking an ETD. The user can easily bookmark an ETD. Bookmarked ETDs can be accessed quickly and easily, without having to remember which list the ETD was saved to.
Log in to an existing profile. If a user is revisiting their past research, the user logs in and can view all their created lists and their respective items with ease.
Evaluation
The study gathered both quantitative and qualitative data to answer the research questions. Quantitative data was gathered by making use of the Usefulness, Satisfaction and Ease of Use (USE) Questionnaire and qualitative data was gathered by asking the participants several open-ended questions about the system. The USE questionnaire contains questions well suited for answering the research questions. The questionnaire consists of 30-items falling into four different categories: Usefulness, Satisfaction, Ease of Use and Ease of Learning. The answer scale is 10-point Likert items. Having both quantitative and qualitative data provided adequate data to test the hypothesis. The questionnaire can be found under project resources on the overview page.
Participants
Thirty participants, who are all current students at the University of Cape Town, evaluated the system. The inclusion criteria for the study was any student who has performed research.
Test Procedure
Participants need a good understanding of the system to provide accurate results. Therefore, users were required to perform a set of tasks on the system before answering the questionnaire. The instruction set was designed to give the participants the best possible understanding of all the features available on the system within a short time frame. The test procedure was designed to be standardised and repeatable to see how different users found the same tasks. The instruction set can be found under project resources on the overview page. After completing the set of tasks on the system, participants answered the questionnaire.
Results
Thirty complete responses were gathered during the experimental phase of the project. However, one response was discarded as illegitimate, so 29 responses were analysed. 62% of the responses were from students in their fourth year of study (Honours) or higher and varying disciplines (mathematics, engineering, finance, computer science, psychology, actuarial science, statistics and medicine).
Overall, the participants found the system useful, easy to learn and use, and satisfied with it. The overall average response from the questionnaire was 8.36. With ten representing "strongly agree" on the Likert scale, the participants overwhelmingly approved of the system’s user experience.
| Categories | Mean | Median | SD |
|---|---|---|---|
| Usefulness | 8.09 | 8 | 1.68 |
| Ease of Use | 8.41 | 9 | 1.88 |
| Ease of Learning | 9.33 | 10 | 1.09 |
| Satisfaction | 8.07 | 9 | 1.90 |
| Overall | 8.36 | 9 | 1.79 |
Proportion of responses for each category
The Figure alongside illustrates the proportion of scores for each category in the questionnaire and the overall proportion of responses. 55.84% (502/899) of the total responses were either 9 or 10 on the 10-point Likert item, showing how most participants strongly agreed with the items in the questionnaire. A negative skew on the data can be seen in the Figure.
Ease of Learning, shown in yellow, was an outperforming category when compared to the overall proportion.
The overall results already indicate participants approved of the system usability and user experience. Of the 899 responses gathered in the questionnaire, only 37 (4.11%) disagreed with an item.
Given the level of detail covered in the USE questionnaire, the low level of disagreement is hugely positive. 829 (92.21%) of the responses were in agreement with an item in the questionnaire.
The original questionnaire was modified slightly for the use of this study. Item 3 ("It is useful") on the original questionnaire was split into two items: "The public lists are useful" and "The private user lists are useful". The custom items offered a more specific statement that relates to the research questions. The two custom items are arguably the most important when trying to answer the research questions.
The item "The private user lists are useful" received an average response of 8.83 with a low standard deviation of 1.20, while the statement "The public user lists are useful" received an average response of 8.79 and a standard deviation of 1.59. Most participants strongly agreed with these two statements, highlighting the usefulness of the two main features provided by this system. The Figure alongside illustrates the frequency of responses for each of the items.
Frequency of item 3 and 4 in the USE questionnaire
In the usefulness section, participants on average agreed strongly that the system both saved them time and helped them be more productive. This point was further strengthened, with 4 participants commenting in the qualitative section that they think the system will save them time by suggesting relevant resources. One of the system's objectives was to provide researchers with a method of finding important material more efficiently than existing research platforms. If this system saves the participants time by suggesting relevant content, it performs tasks more efficiently than other systems and, therefore, meets one of its objectives.
The participants found the system was easy to use. The system being easy to use was an objective during development. One of the highest-scoring items in the questionnaire was item 22 ("I learned to use it quickly"), showing participants found the system easy to grasp.
The analysis of the qualitative data was done using thematic analysis. The first theme was public reading lists. The ability to create public reading lists was regularly mentioned as the most enjoyed feature by participants. Participants enjoyed the public reading feature because it saved them time by suggesting relevant resources. Private reading lists were also a common topic. Half the participants mentioned the private reading lists as their most enjoyable feature. The main reason for this being it provided the participants with a way of organising their research efficiently, which was an objective of the system.
Conclusions
The results from the research provide a clear answer to the research questions. The participants found the system useful and easy to use. It is overwhelmingly clear that the public reading lists provide a useful recommendation feature to help users find relevant content. By suggesting relevant content, researchers save time and are exposed to more information.
It is also clear that private reading lists provide a better user experience than other bookmarking tools. Participants were using inefficient ways to organise and save their research, and the private reading lists provided them with a way of doing so efficiently.