A Chinese military helicopter by Pingtan island across the Taiwan Strait from Taiwan on Thursday
The live fire drills began at 12:00 local time (04:00 GMT) and in several areas were due to take place within 12 miles of the island.
Taiwan said China was trying to change the status quo in the region.
Ms Pelosi made a brief but controversial visit to Taiwan, which China regards as a breakaway province.
The drills are Beijing's main response, although it has also blocked some trade with the island.
The exercises are due to take place in busy waterways and will include long-range live ammunition shooting, Beijing says.
Taiwan says it amounts to a sea and air blockade while the US said the drills were irresponsible and could spiral out of control.
Analyst Bonnie Lin told the BBC that the Taiwanese military would react cautiously but there was still a risk of confrontation.
"For example, if China decides to fly planes over Taiwan's airspace, there is a chance that Taiwan might try to intercept them. And we could see a mid air collision, we could see a lot of different scenarios playing out," she said.
Taiwan said it scrambled jets to warn off Chinese warplanes on Wednesday and its military fired flares to drive away unidentified aircraft over the Kinmen islands, located close to the mainland.
Several ministries have suffered cyber-attacks in recent days, the Taiwanese government said.
Taiwan has also asked ships to take different routes and is negotiating with Japan and the Philippines to find alternative aviation routes.
Japan has also expressed concern to China over the areas covered by the military drills, which it says overlaps with its exclusive economic zone (EEZ).
In response, Chinese government spokeswoman Hua Chunying said Beijing did not accept the "so-called" Japan EEZ.
On Wednesday, China detained a suspected Taiwanese separatist in the coastal Zhejiang province on suspicion of endangering national security, according to local media reports.
Meanwhile China's Ambassador to France Lu Shaye told French TV that after "reunification" with Taiwan, Beijing would focus on "re-education".
China has previously used the term "re-education" to refer to its detention of mostly-Muslim minorities in its north-western Xinjiang region, where human rights groups say more than a million people have been incarcerated.
Mrs Pelosi, the most senior US politician to visit Taiwan in 25 years, made the trip as part of a wider Asian tour.
China had warned her not to travel to the island.
Accusing the US of "violating China's sovereignty under the guise of so-called democracy", Chinese Foreign Minister Wang Yi said: "Those who play with fire will not come to a good end and those who offend China will be punished."
In a statement after the visit, Ms Pelosi said China cannot "prevent world leaders or anyone from travelling to Taiwan to pay respect to its flourishing democracy, to highlight its many successes and to reaffirm our commitment to continued collaboration".
After leaving Taiwan, Ms Pelosi travelled to South Korea, where she met her counterpart Kim Jin-pyo. She is due to visit the Joint Security Area near the border between the two Koreas, patrolled by the US-led UN command and North Korea.
The US walks a diplomatic tightrope with its Taiwan policy. On the one hand, it abides by the "One China" policy, which recognises only one Chinese government, giving it formal ties with Beijing and not Taiwan.
On the other, it maintains a "robust unofficial" relationship with the island, which includes selling weapons for Taiwan to defend itself.
-- Courtesy of BBC Sport