Chapter 324 - 251: Training Data Pipeline

Chapter 324: Chapter 251: Training Data Pipeline

After a night of sorting himself out, Chen Lin finally managed to pull himself out of that peculiar mood.

Nine o’clock sharp.

Chen Lin arrived at the Yanhai Company office right on time.

The moment he pushed open the door, the familiar aroma of coffee wafted over him.

Bai Qian and Pan Si had already arrived. The two were gathered around a large screen, pointing at the densely packed data charts on it.

Shen Yan was sitting at her workstation, a pile of research papers spread out in front of her, her head lowered as she took notes.

Sun Yu, meanwhile, was in the corner, tapping away at his computer, writing some kind of script.

"President Chen is here!"

Sun Yu was the first to notice Chen Lin and immediately jumped up from his chair.

"President Chen." Bai Qian and Pan Si also turned around.

Shen Yan looked up, her eyes meeting Chen Lin’s. She gave a slight nod before lowering her head to continue with her work.

Chen Lin noticed Shen Yan was wearing a light gray knit sweater, the collar revealing a sliver of her pale, slender neck.

"..."

Chen Lin averted his gaze and walked over to the large conference table in the middle of the office, taking a seat.

"Everyone’s here. Let’s have a meeting."

For the next half hour, Bai Qian and Pan Si took turns reporting on their work progress over the past few days.

There was one central point—

The test results for the YanHai-1.5B model were in, and every metric had far exceeded expectations.

"To put it in a single sentence—"

Bai Qian pushed up his glasses, unable to hide the excitement on his face.

"We used one-fifth of the parameters and one-tenth of the training data to match or even surpass the mainstream mid-sized models on the market."

"In the context of the entire AI industry, this result is nothing short of explosive."

"Now, we need to consider our next step."

Pan Si took over:

"According to the original plan, the next step is to start training the 30B parameter model."

"But..."

He paused, his expression growing serious.

"Training a 30B model is on a completely different scale than a 1.5B model."

"Even with the boost from President Chen’s new architecture, and with VRAM usage already reduced by sixty percent, a full training run would, conservatively, still take at least a month."

"And there’s another problem—"

Pan Si looked at Chen Lin.

"Data."

"The corpus I prepared earlier is fine in terms of quality, but its coverage in certain vertical domains is insufficient."

"For example, highly specialized fields like medicine, law, and finance."

"If we want the 30B model to perform well in these areas too, we need more high-quality, specialized data."

After listening, Chen Lin nodded slightly.

He had thought of this problem before.

"Regarding the data, I mentioned it to Vice Minister Liao from the Ministry of Technology," Chen Lin said, his tone even.

Bai Qian and Pan Si both perked up, their eyes glinting with anticipation.

’If we could get national-level data access, that would be a dream come true!’

Even a fraction of the high-quality data strictly guarded by major government agencies would be enough to train several models.

However—

"Vice Minister Liao’s reply was—"

Chen Lin paused, looking at their hopeful eyes, feeling a bit reluctant to continue.

But he had to say it.

"First, the data group isn’t under the Ministry of Technology’s jurisdiction."

"Second, various regions across the country are currently setting up their own local data groups. The Huaxia Data Group hasn’t been formally established yet."

"So, for now, they can’t grant us national data access."

As he finished speaking,

The conference room instantly fell silent.

Bai Qian and Pan Si exchanged a look, their expressions dejected.

"Sigh..."

Bai Qian sighed, taking off his glasses and rubbing the bridge of his nose.

"I knew it wouldn’t be that simple."

"Of course, they wouldn’t just hand over something like national data access."

Pan Si also nodded. Although his face showed some disappointment, he quickly regained his composure.

"Alright, since the official channels are a no-go, we’ll have to rely on ourselves."

He pushed up his glasses, his tone turning serious.

"Actually, I had a backup plan in mind."

"What plan?" Chen Lin asked.

"Building a training data pipeline."

Pan Si stood up, walked to the whiteboard, and picked up a marker to start drawing.

"The whole process has three steps—"

"Step one, scrape data from the internet. This is essentially just a slightly more complex web scraper. There’s no real technical difficulty."

"Step two, clean the data. This step is crucial."

He drew a box on the whiteboard and wrote "YanHai-1.5B" inside.

"We can use our trained 1.5B model as a foundation to quickly assess and clean the raw data we scrape."

"The 1.5B model might not have a large number of parameters, but its basic semantic understanding and quality assessment abilities are more than enough."

"Using it to filter out low-quality, repetitive, and meaningless content will be dozens of times more efficient than manual sorting."

"Step three, feed it to the model for training. This step isn’t much different from before; we just send the cleaned data into the training pipeline."

Pan Si put down the marker and turned to Chen Lin.

"Once the entire pipeline is set up, it will be like an ’automated data factory’."

"As long as the servers are running, it will continuously crawl data from the internet, clean it, and generate training corpora."

"Although it might still lack specialized data in vertical domains, it will at least solve the problem of ’quantity’."

Chen Lin listened, quickly evaluating the plan’s feasibility in his mind.

’From a technical standpoint, the plan is feasible.’

’Using a smaller, pre-existing model to help with data cleaning is common practice in the industry.’

’It’s just that most companies have mediocre small models, so the cleaning results are usually just as mediocre.’

’But Yanhai is different.’

’Their YanHai-1.5B was trained using that "Magic Architecture" of mine, after all.’

’Despite having only 1.5 billion parameters, its actual performance is close to, if not better than, the 7B models on the market.’

’Using this thing for data cleaning would definitely produce great results.’

"This plan..."

Chen Lin pondered for a few seconds, then nodded.

"Doable."

A smile instantly spread across Pan Si’s face.

"However—"

Chen Lin’s tone shifted.

"Before we officially start training the 30B model, I think we should build this data pipeline first."

"For one, it’s to stockpile more training data for the 30B model, and for another, it’s to verify that this whole process can run smoothly."

He looked at Pan Si.

"You’ll lead the development of this pipeline."

"As for manpower, you can allocate resources as you see fit."

Pan Si immediately nodded. "No problem."

Chen Lin then turned to Bai Qian.

"Dr. Bai, after the pipeline is built, you’ll be in charge of training the 30B model."

"Got it." Bai Qian nodded as well.

Finally, Chen Lin’s gaze fell on Shen Yan and Sun Yu.

"You two, for the time being, will follow Pan Si and help with the pipeline development."

Shen Yan gave a soft "Mm," to show she understood.

Sun Yu, however, was visibly excited.

"Great! I can finally do some real work!"

"Those data cleaning scripts I wrote before can finally be put to use!"

Chen Lin watched Sun Yu’s eager expression, and the corner of his mouth twitched.

’This guy... He’s certainly enthusiastic, but I’m not sure if he’s reliable.’

’Then again—’

’Sun Yu’s fundamentals might be a bit weak, but he’s made obvious progress studying under Pan Si these past few months.’

’He went from knowing zero Python to being able to write simple scripts on his own.’

’His coding style is a bit... uh... rustic?’

’But at least his code runs, which makes him a usable asset.’

"Alright, that settles it then."

Chen Lin stood up and announced the end of the meeting.

"On the pipeline development front, everyone give Pan Si your full cooperation."

"If you have any problems, come find me anytime."

After the meeting, everyone got busy.

Pan Si gathered Bai Qian, Shen Yan, and Sun Yu for a small technical meeting to discuss the pipeline’s specific implementation. Chen Lin listened in from the side; his own Python skills were quite decent now, and he’d have to help out with the development when the time came.

Time slipped by quietly.

In the blink of an eye, it was noon.

"Boss, we’re heading out for lunch~"

Sun Yu’s voice came from the doorway.

Chen Lin looked up and saw Sun Yu pulling Bai Qian out the door.

Pan Si, on the other hand, had brought his own lunchbox and had already started eating at his desk.

Shen Yan...

Chen Lin glanced around and found Shen Yan still sitting at her workstation, staring at her computer screen with her brow slightly furrowed, as if pondering some problem.

"Not going to eat?"

Chen Lin asked.

Shen Yan snapped out of it and looked up at Chen Lin.

"I... I’ll go in a bit."

Her voice was soft.

"There’s a problem with this piece of code. I want to figure it out first."

Chen Lin stood up, walked over to Shen Yan’s side, and glanced down at her screen.

Displayed on it was a piece of Python code, part of the data scraping module.

"Where’s the problem?"

"Here."

Shen Yan pointed to a loop on the screen.

"I’m trying to use multithreading to improve scraping efficiency, but for some reason, it freezes whenever I run it."

Chen Lin looked for a few seconds and quickly found the problem.

"Your lock here isn’t being released correctly."

He pointed to one of the lines of code.

"Look here. After you acquire the lock, if an exception occurs, the program jumps directly to the `except` block."

"But you don’t release the lock in the `except` block, so the other threads just keep waiting, and eventually, the whole thing freezes."

Shen Yan followed his finger, and realization dawned on her.

"So that’s why..."

She looked up, her gaze meeting his.

They were very close.

So close that Chen Lin could smell the faint, fresh scent on Shen Yan.

"..."

"..."

Their gazes met, and both seemed to freeze for a moment.

"Ahem."

Chen Lin was the first to recover. He took a step back and cleared his throat.

"It’s not a big deal. A small change will fix it."

"While I’m at it, I’ll write up a template for exception handling for you. You can just apply it directly to similar situations in the future."

"O...Okay."

Shen Yan lowered her head, the tips of her ears turning slightly red.

Chen Lin found a chair, sat down next to Shen Yan, and began to help her fix the code.

The two of them sat side by side like that, one typing on the keyboard, the other watching from the side.

The only sounds in the office were the tapping of the keyboard and the occasional chewing sounds from Pan Si as he ate.

The atmosphere was quiet and cozy.

1:30 PM.

Chen Lin put down his coffee cup and picked up his phone to check the time.

It was time for his meeting with Fu Yangqing.

He stood up and called out towards Pan Si.

"Dr. Pan, I have to step out for something this afternoon. You guys carry on."

"Okay, President Chen," Pan Si replied without looking up.

Shen Yan also looked up, a hint of a question in her eyes.

Chen Lin nodded at her as a form of greeting, then walked out of the office.

2:00 PM, sharp.

The fourth floor of the Yannan University cafeteria.

This floor usually wasn’t crowded, consisting mainly of some Western-style light meals and cafes.

Chen Lin found a corner seat and ordered two coffees.

Before long, a lean, stern-faced man walked over.

It was none other than Fu Yangqing.