US Citizen and Immigration Services Adopts Event-Driven Data Architecture

US Citizen and Immigration Services Adopts Event-Driven Data Architecture

US Citizen and Immigration Services, which is a subdivision of Homeland Security, has partnered with Confluent to build an event-driven data architecture. This will provide live data streams to third party organizations that rely on US CIS data to offer immigration services. The partnership was unveiled at Confluent’s user event in Austin last week, where, US CIS CTO, Rob Brown, explained that the new data arcitecture will share data in real time with other US government departments, such as the Customs and Border Patrol, so they can quickly see an applicant’s whole immigration journey and, therefore, be able to provide them better service. He said:

I like to think that we provide the American Dream for folks. We have a lot of people that rely on immigration benefits, but also really on the humanitarian side, we work a lot with refugees. We try to help a lot of these folks and try to give them lawful immigration rights in the United States. We’ve got about 20,000 employees and we’ve got about 350 offices around the globe. 

So, we’ve got a lot of people, a lot of offices, doing a lot of work, and a lot of those adjudication activities rely on a lot of data and information from a lot of other organizations – from our friends at Customs and Border Control, to our colleagues over at ICE, and other folks like the Department of State.

US CIS is not new to data sharing; this sharing, however, was previously done upon request and through a compex data architecture. A shortcoming of this architecture was the inability to share live, native data streams. Data had to be shared as a complete dataset, which made it cumbersome to analyze and manage. But to move to a streaming architecture, US CIS had to first build an in-house system since most PaaS and SaaS cloud vendors are not FedRAMPED. Brown added:

We traditionally had a spaghetti mess of how we were sharing data with various other business units. We’ve had to rely on spreadsheets and email exchanges. So taking a step back over the past few years, we’ve really been trying to take note of: what does that mean? How do we start to centralize a lot of our integration activities? What new technologies exist? And how can we start to deploy them not just internally for ourselves, but also externally for our partners too? Both in the consumption and the production of data.

At the moment, US CIS is using REST APIs, and is in the process of shifting to Confluent. The organization plans to phase the transition from REST to Data in Motion by transitioning, first, to Direct Access (S3).

This shift did not happen overnight; the organization had taken a domain driven approach as far back as 2016 which matured over the years with the adoption of microservices surrounding the data system. This model will eventually be adopted by other Federal organizations, including the Customs and Border Patrol (CBP), in an attempt to consolidate data across organizations.

Post a Comment