Backend Development for MyAdvisor
Building the server-side infrastructure and database architecture for UCT's virtual student advising platform
As the backend developer and database manager for MyAdvisor, I was responsible for implementing all server-side logic and ensuring the seamless functioning of the system behind the website. My work included designing and managing the database to organize unstructured handbook data into a structured format, as well as developing the API that enables the frontend to interact with this data.
I handled all logic performed on data traveling to and from the backend, including validating and formatting API responses for the frontend in the desired format, as well as validating and formatting frontend requests that needed to be stored in the database. I also developed the admin interface, which enables staff members to create, read, update, and delete any information stored in the database, thereby updating the information on the site.
User testing was conducted with eight participants, all of whom found the admin page intuitive and effective for managing site content. The backend is built using the Django REST Framework, making it highly modular and scalable. Each component operates independently, allowing for easy customization, extension, and maintenance. This modular architecture ensures that the backend can be efficiently adapted to meet evolving project requirements.
Primary Objectives
Easy Data Access
Ensure easy access to all MyAdvisor data for the frontend and in the correct format
Data Centralization
Centralise all academic data in all the different UCT faculty handbooks
Staff Management Interface
Allow staff to update any UCT academic data in one place
Data Consistency
Maintain data consistency throughout the website
Technologies Used
Python
Versatile, high-level programming language known for its readability, simplicity, and broad use in web development, data analysis, and automation. Python's extensive ecosystem made it ideal for rapid backend development and data processing.
Django
High-level Python web framework that enables rapid development of secure, scalable, and maintainable web applications by providing built-in tools for databases, authentication, and user interfaces. Django's "batteries-included" philosophy accelerated development.
PostgreSQL
Powerful open-source relational database management system known for its reliability, advanced SQL compliance, and strong support for complex queries and data integrity. PostgreSQL's robust features ensured data consistency and scalability.
Django REST Framework
Powerful toolkit for building Web APIs in Django. Provides serialization, authentication, and browsable API features that made it easy to create a robust, well-documented API for the frontend to consume.
System Requirements
Storage
The backend must store all system information required by the frontend, including course data, degree requirements, faculty information, and user accounts.
Authentication & User Management
Register students in the database by processing a new student's data received from the frontend. Implement secure authentication and session management for both students and administrators.
Academic Information Access
The backend must supply the frontend with lists of data from the database, including all faculties, departments, programs within each faculty, degrees or majors within each program, and courses within each degree, major, or department.
Student-Advisor Communication
Take a question asked by a student on the frontend, find an advisor to answer that question and store all the question's information in the database. Implement routing logic to assign questions to appropriate advisors.
Student Help Resources
Provide the frontend with lists of all FAQs, FAQ sections, glossary terms with their definitions, and details of all advisors from the database. Ensure efficient querying and filtering capabilities.
Admin Page
The system must include an administration interface that enables designated administrative staff to manage database content through a secure and user-friendly web interface. Support full CRUD operations for all data models.
Challenges & Solutions
Data Normalisation
CHALLENGE
Converting handbook and textbook data into a format suitable for a relational database. Prerequisites could not be fully normalised due to their complexity and variability.
SOLUTION
Careful schema design with prerequisite information stored as plain text. This allows the chatbot to access prerequisite information while maintaining database integrity.
Bulk Data Insertion
CHALLENGE
The sheer volume of data in the handbooks (thousands of courses, degrees, and programs) made manual entry impractical. Even using Python scripts, populating the database was time-consuming and error-prone.
SOLUTION
Used AI to drastically reduce manual labour. After several iterations, AI created bulk SQL statements that perfectly matched the structure of degrees, majors, and courses in the handbooks.
Complex Degree Structures
CHALLENGE
The degree structures described in the handbooks are highly complex and very difficult to apply to a database schema.
SOLUTION
Introduced two additional tables: course group and parent course group, which effectively solved the issue and allowed for flexible degree structure representation.
Electives Management
CHALLENGE
Electives presented additional challenges. Some degrees require no electives, while others include many. An elective represents a choice among several courses, making them difficult to model and store.
SOLUTION
Introduced wildcard courses as placeholders for elective choices in specific departments and years. This enabled electives to be stored as individual courses, simplifying the schema without causing significant issues.
Reflections & Learnings
Significance of the Backend
Before this project, I did not realize just how vital the backend is to a project. While working on it, I learned that it is a considerable task and gained a massive understanding of how modern backend systems work.
Technical Growth
In building the backend using Django and PostgreSQL, I learnt how to design and structure a complicated database, create and manage APIs, handle data securely, and connect the backend to a frontend application. Working with Django taught me the principles of how to get data from the database to the frontend and vice versa, as well as authentication and server-side logic. PostgreSQL enhanced my skills in relational databases, schema design, and efficient data querying.
Python and SQL Proficiency
Through this process, I also strengthened my skills in Python and SQL, which are foundational for backend development. Python for writing clean and maintainable code, and SQL for effective data management and manipulation. These skills are essential because they enable the creation of reliable systems that store, process, and serve data to users, forming the core of nearly every modern web application.
Collaboration and Teamwork
I learnt the importance of communication, coordination, and version control in ensuring that different parts of a system integrate smoothly. Working in a team teaches how to use tools like Git and GitHub for managing code collaboratively.
Testing Methodologies
I learned many new ways to test code and API calls, as you cannot test them the regular way by running the code. Specifically, I learned how to use Postman to test my API calls and review the JSON responses I would receive and use for each call.
Supporting Materials
Final Paper
Complete documentation of the Backend development process and technical implementation.
literture Review
Initial project proposal outlining goals, methodology, and expected outcomes for the Backend component.
Source Code
Complete Backend source code including all components and configuration files.
Live Demo
Access the live MyAdvisor application to experience the frontend implementation firsthand.
(External, non-permanent link)
Credits & Acknowledgements
Project Team
- •Ethan Allies - Frontend Development (React.js, Next.js, Tailwind CSS, TypeScript)
- •Dylan Howard - Chatbot Development (NLP, Hugging Face Transformers)
- •Mitchell Flanegan - Database Development (Django REST Framework, PostgreSQL)
Supervision
Aslam Safla, Department of Computer Science, University of Cape Town
Special Thanks
am deeply grateful to my supervisor, Prof. Aslam Safla, for his invaluable guidance and support throughout the development of MyAdvisor, and to the five UCT advising staff and eight Computer Science students who provided critical feedback during testing, enhancing the backend’s usability and data consistency. Special thanks also go to the UCT IT Services team, particularly Craig Balfour, for their technical assistance with server access and PostgreSQL troubleshooting.