Unsurprisingly, infinite was a common them for them. They, in fact, considered different sizes of infinity which they called enumerable, innumerable, and infinite, previewing transfinite cardinals by nearly 2000 years.
The Mayans also had a fascination with large numbers (although not quite that large), and their use in calendars (which were much more mathematically well-constructed than any have been in the West, surely than they were during the first millennium CE, when the Mayans were at their peak). The Mayans kept careful astronomical observations and devised their numeration system around their calendar and astronomical needs.
The Incans wanted a way to deliver recorded information across great distance before they had developed writing. They did so by tying a sequence of knots into linked strings. These were easily transported by runners who delivered them across difficult mountainous terrain. These quipus are from 1400-1550 CE.
There were over a million people living in Cairo of ancient Egypt, often working together as a complicated society. The papyri we have sometimes concern the computations necessary to maintain their many projects. For example, the Reisner papyrus appears to be a set of books for a construction site. They include a list of employees, as well as calculations of volumes and areas. A typical computation is determining the number of workmen needed to excavate a tomb. These documents are from 2000 - 1500 BCE.
Here are some views of the Rhind
Papyrus - to give you a sense of what this relic actually
The Moscow Papyrus
Here is a little bit from the Moscow Papyrus.
Some problems from the Rhind and Moscow - translated. More detailed version. More sources.
The Berlin Papyrus
The Mesopotamian culture was one of the first agricultural societies. They kept records of this by marking in mud with a stylus and then baking into tablets. These have the advantage of preserving very well. Many of their problems were about canals. They also seem to have an interest in math problems for curiosity sake as well. The works we have are from a range of time from 2500 to 1600 BCE. Suffice to say - very long ago.
The Nine Chapters of the Mathematical Art is the canonical ancient Chinese textbook (held in the same esteem as Euclid's Elements). It has many problems and sections on diverse practical and instructional topics. It also has many editions through history with a tradition of several different commentators making additions. This work is from about 200 BCE to 200 CE. The "Pythagorean theorem" is well known in many cultures, here we see it was familiar in China in about 500 BCE.
of Zhoubi suanjing
An image of nine chapters. Text from nine chapters.
Contents of nine chapters (excuse the Wikipedia link - I do have this in a print source but this way I don't need to scan it here).
Yang Hui's triangle
The Indians were also familiar with the "Pythagorean theorem" at least two hundred years before Pythagoras (~600 BCE). Much of their geometry work was used for temple building in ancient India. Indian work with trigonometry in about 500 CE is motivated by astronomy and is mostly computational (as is most early trigonometry). The Lilivati from Bhaskara represents an overview of mathematical achievements at the time throughout their history (~1150 CE).
Early Islamic culture in the middle-east was a varied cultural mixing ground, with regular travelers and commerce from Europe, Africa, and the far East. It lead to great wealth and a respect for diversity, which included an attempt to gather and develop ideas from each culture they contacted. Inspired by numeration from India, and commercial record-keeping needs, in the 9th century, Algebra is developed independently from the European influence in Geometry. Islamic geometry was also often algebraic, but also included trigonometry (largely for needs in astronomy). One catalyst for geometric work came from creating architectural art - which was highly geometric due to a prohibition of depiction of living objects.
Some from al-jabr.
Early decimal point
Abu'l Wafa's finger reckoning was an influence on things like this.
al-Haytham on geometry.
Great mosque - detail.
al-Khayyami on the cubic
al-Mun'im's arithmetic triangle
al-Biruni's qibla problem
There is much left to learn. For me one big area is sub-Saharan Africa.
The Crest of the Peacock: Non-European Roots of Mathematics, George Gheverghese Joseph, Princeton University Press, 2011
Mathematics in Historical Context, Jeff Suzuki, Mathematical Assocation of America, 2009
A Contextual History of Mathematics, Ronald Calinger, Prentice-Hall, 1999
A History of Mathematics: An Introduction, Victor J. Katz, Addison-Wesley, 1998
The History of Mathematics: A Reader, John Fauvel and Jeremy Gray, eds., Open University, 1987
Classics of Mathematics, Ronald Calinger, ed., Moore Publishing, 1982