Problem

The main objectives of the longitudinal impact study were centered on capturing the broader and deeper role the SFSG played in participants’ lives. First, we aimed to capture the qualitative impact of the guide across multiple categories of quality of life, such as housing stability, financial and job security, access to health services, and education. Second, the study sought to map the user journey in detail, both before and after interacting with the SFSG, to identify pain points not only within the platform itself but also in the wider process of trying to access social services in San Francisco. Third, we wanted to better understand user behavior after leaving the guide — for example, whether they contacted a service, whether those interactions were successful, and whether they chose to return to SFSG or not. Finally, the study set out to identify opportunities for ShelterTech to improve the product experience and expand its support, particularly for users facing systemic barriers such as lack of technology access, transportation, or social support networks. By framing these objectives around both lived experience and actionable design opportunities, the study created a clear path for tying research insights directly into product development.

Information visibility: Users, especially non-digitally native individuals, had difficulty navigating or even discovering resources on the site.
Accessibility barriers: Those without steady internet access or who relied on nonprofit staff as intermediaries could not benefit fully.
Unclear program impact: ShelterTech lacked a long-term understanding of how SFSG was being used over time, what gaps existed, and how its resources truly affected people’s daily lives.
Without structured feedback, ShelterTech risked building solutions that didn’t match real community needs or evolving circumstances.

Solution

The research design was longitudinal, combining both exploratory and evaluative approaches in order to gain a holistic view of user experiences over time. Conducted between February 2024 and May 2025, the study involved three major rounds of semi-structured interviews with the same group of participants, as well as completion follow-ups to capture longer-term shifts. Participants were selected based on clear inclusion criteria: they had to be San Francisco residents who had used the SFSG at least once to search for resources for themselves, rather than on behalf of someone else. To ensure diversity and account for attrition, the study targeted 12 to 15 participants overall. The qualitative interviews, conducted via phone, Zoom, or in person, followed a structured but flexible guide that explored participants’ current circumstances, previous life events leading to their present situations, experiences using SFSG and other social services, and their hopes for the future. These questions were open-ended to encourage authentic storytelling and to avoid introducing bias. In parallel, quantitative surveys with Likert-scale questions were used to measure baseline needs, track changes in quality-of-life categories over time, and allow for comparisons across participants. Recruitment was carried out through community channels such as the SFSG website, prior survey lists, nonprofit partners, and direct outreach, and participants were compensated with $75 per interview plus a completion incentive. The process followed a clear cycle: proposal development and literature review, recruitment, baseline interviews, interim analyses, midyear adjustments to research tools, additional interview rounds, and final synthesis and recommendations.

This combines both qualitative and quantitative methods to capture a holistic view of community needs. On the qualitative side, recurring semi-structured interviews with housing-insecure individuals, nonprofit staff, and job seekers are conducted, supported by personas and journey maps to illustrate lived experiences. When needed, wireframes and MVPs are tested to gather usability feedback. To reach those with limited digital access, my research team partner with nonprofits like Downtown Streets Team. On the quantitative side, surveys using Likert scales and predefined categories help measure satisfaction, ease of use, and overall impact at scale. Insights are then synthesized through workshops and share-outs with product and executive teams, ensuring that findings directly inform ongoing design and product decisions.

Outcomes

The research generated a range of important insights that shed light on the realities of navigating social services with the help of SFSG. Participants’ housing situations were often precarious, and financial or employment instability was one of the most common root causes of homelessness or housing insecurity. Many participants reported significant challenges in their overall well-being, citing not only physical needs like food and healthcare but also emotional struggles such as stress, anxiety, and social isolation. Technology access emerged as a double-edged sword: while nearly everyone owned a mobile phone, many were constrained by limited data plans, unreliable connectivity, or low digital literacy, which reduced the usefulness of online resources like SFSG. Transportation also emerged as a consistent barrier, as long distances or lack of reliable transit often prevented people from following through on resource connections. When it came to the SFSG itself, participants generally valued having a centralized directory of services, but they expressed that the guide often fell short in providing personalized, actionable pathways. While some found it useful for initial discovery, many struggled with what to do next or whether the services listed were accessible to them given eligibility criteria, documentation requirements, or other external obstacles.

The impact of this study extended beyond documenting challenges; it directly shaped the way ShelterTech approached its product strategy and design. The findings helped guide the product roadmap by highlighting the need for more personalized assistance within SFSG, such as tailoring results to individual eligibility or offering clearer next steps. They also reinforced the importance of designing with accessibility in mind, not only in terms of interface usability but also considering the realities of users with limited data access or low digital literacy. The personas and journey maps produced as part of the synthesis process became practical tools for product and design teams, grounding decision-making in real user experiences rather than assumptions. Furthermore, the research broadened ShelterTech’s understanding of adjacent problem spaces, leading to new initiatives and interests in areas such as job assistance, the effects of COVID on housing instability, and increasing the visibility of resources like utility support. By combining qualitative storytelling with quantitative measurement, the study produced insights that were both empathetic and actionable, ensuring they could translate into tangible improvements for users.

Lessons Learned

One of the most important lessons from this work was the centrality of trust and psychological safety in conducting meaningful research with vulnerable populations. Participants were far more willing to share their stories openly when they felt respected and assured of confidentiality. Another lesson was the power of open-ended questioning, which allowed participants to surface challenges and experiences that might not have been anticipated in advance. The combination of qualitative interviews with quantitative surveys also proved essential, as it balanced individual stories with a broader picture of trends and impact. Finally, the study underscored the fact that accessibility must be considered holistically: it is not enough to make a digital platform usable on a screen if systemic barriers like transportation, literacy, and connectivity are left unaddressed.

Next Steps

Looking ahead, the study will continue with additional interview rounds to capture changes over the full year-long period, followed by a final analysis and report release in 2025. My team plans to refine the interview guide based on midyear findings and explore prototype concepts for features such as personalized assistance flows or offline resource navigation. Another key next step is building stronger partnerships with nonprofits and case managers to expand adoption of SFSG among users who may be less digitally native and require more hands-on support. Ultimately, the insights from this research are intended not only to refine the SFSG but also to help ShelterTech continue evolving its role as a technology partner that empowers unhoused and housing-insecure individuals to navigate complex systems and improve their quality of life.