The Silicon Valley giant has chosen the landmark building as the first UK site for a groundbreaking initiative working with businesses called Google Digital Garages.
Part of the library, which recently saw its hours slashed in council budget cuts, will be given up for a new area offering the digital training for the skills small and medium-sized businesses need to thrive.
It will open its doors on Monday.
Coun Penny Holbrook, Birmingham City Council's cabinet member for skills, learning and culture, said Google's arrival was a shot in the arm for the library after 100 staff were axed and opening hours cut.
She said: "The council has been facing difficult times and anything we can do to operate differently and work with other people is to be welcomed.
"This collaboration between the Library of Birmingham and Google is fantastic news, following hot on the heels of our partnership with the British Library.
"This shows that major organisations not only want to come to Birmingham, they want to work with the city council and the Library of Birmingham."
The Google Garage, the first of five such ventures following a pilot project in Leeds, will operate in Birmingham until at least the end of the year.
Coun Holbrook said Google's arrival was the latest in a number of initiatives designed to make full use of the library, which sets the authority back £2 million a month.
She added: "There has always been a plan for how we would try to recover – part of it is the partnership with the British Library and part of it is Google.
"We are looking at how we use the space to the best of our ability. It is really important to keep the public library function but we need to think about more creative ways to use it that could expand its offer."
The Digital Garages are part of a multi million-pound initiative being rolled out by Google which will ultimately see 200,000 UK businesses receive free digital masterclasses, to include one-to-one mentoring sessions and a digital 'tune-up'.
Google is also planning to help develop future digital talent via its garages by organising coding training sessions for 25,000 teachers.
The firm has pledged to teach small businesses vital skills for the digital age, helping them to use the power of the internet to reach more customers and increase productivity.
Google says this expert help will serve as a "growth engine" for SMEs and would-be entrepreneurs giving them the power to transform themselves into the digital savvy businesses of tomorrow.
Training will cover designing and building websites, social media, analytics, online video, Google Adwords and Google Tools.
Eileen Naughton, Google's managing director for UK & Ireland, said: "It can be easy to think that technology is just for start-ups, but we know it can bring real growth to all businesses.
"So, with fewer than 30 per cent of SMEs having an effective online presence, we're working to help jump-start the other 70 per cent.
"At a time when there is pressure for the UK to boost productivity to contribute to real wage increases, we believe Google can be a real growth engine that helps deliver these gains."The US giant, which has been under fire in Europe for its dominance in search and other digital services, said last month it would train a million Europeans in internet skills by 2016, including building an online hub to support small enterprises.
Speaking about the benefits for Birmingham, Coun Holbrook said: "The Digital Garage will help businesses in Birmingham and the wider region maximise their use of digital technology and they will also be able to use the library's meeting spaces for seminars.
"Being digital savvy is absolutely crucial in today's business environment.
"Given most of the new digital jobs in the future won't be in the traditional IT and computing fields, but will be embedded across all sectors of industry, we need to develop the talent and entrepreneurial skills of businesses to ensure they have the right blend of digital skills at all levels."
Other partners in the Digital Garage Birmingham include Marketing Birmingham, Greater Birmingham Chamber of Commerce and the Greater Birmingham and Solihull Local Enterprise Partnership.
The project will also see an educational initiative in partnership with Code Club Pro, Computing at Schools and Raspberry Pi. Workshops and events for teachers will be held at the library and in local schools.