Training NanoZhGPT: Building a 100M Parameter Chinese Language Model

A few weeks ago, I came across a Zhihu article where someone trained a 3M parameter Chinese LLM with decent results. That sparked an idea: why not try training my own? What started as a modest 10M parameter goal eventually evolved into a 100M parameter model.

2024末的芙宁娜

Receive Interview Invitation from St Catz

After some help from GPT-4, I finally began working on the CBuer language compiler. Initially, I was hesitant about breaking things apart. I wanted to put most of the frontend, such as the Lexer and Parser, into a single state machine. It seemed simpler at first, but it quickly became a nightmare to maintain. Every change meant sifting through a tangled mess, and extending functionality was getting harder by the day.

Eventually, I embraced a modular approach, separating each of the compiler components into distinct modules. By breaking down the components into manageable parts, everything started falling into place. Each module had a well-defined responsibility—Lexer, Parser, and the subsequent stages were their own entities, communicating through clear interfaces.

This separation brought several benefits. Writing new features became more straightforward, debugging became easier, and more importantly, making changes didn’t feel like defusing a bomb anymore. I could work on the Lexer without worrying about breaking the Parser, and vice versa. Each piece of the puzzle had its own rules and scope, reducing the complexity of the overall system. It’s like building with LEGO blocks rather than trying to mold a monolithic, shapeless lump of clay.

One major lesson I learned during this journey is the power of incremental programming. It wasn’t about building everything perfectly right away, but rather developing each module iteratively. As I grew more familiar with the project, my development speed increased as well. Incremental changes became second nature, and adding new features felt like stacking another brick rather than reconstructing a tower.

Of course, AI assistance played a significant role in this transformation. I found that, while AI is great for a lot of tasks, it struggles even after moving to a modular approach, especially when the problems are deeply interconnected. Despite the modularity, GPT-4 often faced challenges in addressing the complexities inherent in such systems.

Modularity has truly changed the way I approach programming. It made my work more structured and efficient, but also much more enjoyable. Taking this step towards embracing separation and incremental progress was a game-changer for developing the CBuer compiler.

Finally, I’ve started writing my book, “Miss from Wuloong.” I’ve wanted to write a novel for a long time, but never had the chance. My writing skills aren’t great, and I’m a slow writer, which made it really hard to get started. Thanks to the amazing GPT-4 canvas, I finally have ideas and can get some help with the writing process, so my limited literary skills won’t hold me back anymore.

Even though I’m using GPT, it still took quite a bit of time. Honestly, GPT isn’t quite powerful enough just yet—there are many fixed patterns in the story, and I have to keep adjusting and refining them repeatedly. But despite all that, I’ve finally started writing, and that feels fantastic. I hope it turns out well, and I’m excited to see where this journey takes me.

Writing has always been a dream of mine, but it felt like an unattainable goal for a long time. I would often think about different story ideas and characters, but I lacked the confidence to put them into words. It’s easy to get discouraged when you compare yourself to professional authors, but I realized that everyone has to start somewhere. The important thing is to just get the words down, no matter how imperfect they are.

With “Miss from Wuloong,” I want to bring to life the beauty of Wuloong and the experiences of the characters who live there. It’s a story about growth, love, and the challenges that come with change. Wuloong is a place filled with cultural richness and natural beauty, and I hope to capture that in my writing. My protagonist is someone who embodies both strength and vulnerability, and I want readers to connect with her journey on a personal level.

There are so many aspects of writing that I’m learning along the way—plot development, character arcs, pacing, and even just the discipline of writing regularly. Using GPT helps me overcome writer’s block and gives me new perspectives on how to approach certain scenes. But it’s still up to me to make the final decisions, add my own touch, and make the story uniquely mine. I think that’s the most rewarding part: knowing that, in the end, it’s my story, shaped by my experiences and imagination.

It’s been a challenging start, but I’m optimistic. I believe that if I keep pushing forward, I can create something meaningful. I’m grateful for this opportunity to finally bring my ideas to life, and I hope that one day others will enjoy reading “Miss from Wuloong” as much as I enjoy writing it.

From 2024/12/14:
I think this novel will just end. Data does not good.

We’re thrilled to announce the release of AkashaDDNS v4.0.2, a significant update for the dynamic DNS solution brought to you by OpenSumeru. This version represents a major step forward in reliability, performance, and simplicity, making dynamic DNS management easier than ever for everyone.

If you have experience working with strings in C++, you’re likely familiar with the many challenges and complexities involved. Managing buffer sizes, ensuring efficiency, and maintaining code readability can make handling strings a tedious task. That’s why I’m excited to introduce my latest project, TabNahida/TString: a new approach to string handling in C++ that aims to simplify your work.

Using PrizmSDK to Build Programs for Casio Graphing Calculators

Introduction

If you’re a developer who loves coding or is interested in running custom programs on Casio graphing calculators, you may have heard of PrizmSDK. PrizmSDK is a software development kit (SDK) designed for developing programs that run on Casio graphing calculators, such as the fx-CG series. In this article, we will walk you through how to use PrizmSDK, which integrates libfxcg, mkg3a, gcc, and binutils into a complete SDK toolchain.

Various Mathematical Constants to 1 Billion Digits

This website collected mathematical constants including:

• Square root of 2 (Sqrt(2))
• Square root of 3 (Sqrt(3))
• Golden ratio
• e
• Pi
• Natural logarithm of 2 (Log(2))
• Natural logarithm of 3 (Log(3))
• Natural logarithm of 10 (Log(10))
• Apéry’s constant (Zeta(3))
• Lemniscate constant (Lemniscate)
• Catalan’s constant (Catalan)
• Euler-Mascheroni constant (Euler’s Constant)

up to 1 billion digits.

Automate Hexo Deployment to GitHub Pages with GitHub Actions: A Comprehensive tutorial

Deploying a static website generated by Hexo to GitHub Pages can be seamlessly automated using GitHub Actions. This tutorial will guide you through setting up an automated workflow that:

1. Generates your Hexo site every time you push changes to your repository.
2. Deploys the generated static files to the docs folder of a separate GitHub Pages repository, enabling continuous updates to your website.

By the end of this guide, you’ll have a fully automated CI/CD pipeline for your Hexo blog, ensuring your website remains up-to-date with your latest content without manual intervention.