Super interesting insights into building with LLM apps in the co-pilot explorer blog

The engineering effort that goes into choosing the right context, fine-tuning and telemetry is immense!

some details:

To measure accuracy co-pilot doesn't just look at if you accepted a suggested piece of code.

You might accept but edit.

So they actually check if some version of that code (within some edit distance) is still present after 15s,30s, 2min, 5min and 10mins!

They stuff the context of the prompt with carefully selected snippets from 20+ recently opened files, using language and recency to help work out what to include.

For each file they split the doc into windows and check for similarity with the code you've just written.

Latency really matters,so you can't use the largest codex models. Everything is built on custom finetunes of smaller models.

Nvidia and Intel team up on chips for metaverse workstations

Those workstations will be launching with anywhere from one Nvidia graphics processing unit (GPU) to four in a system powered by Intel’s latest Xeon W and Intel Xeon Scalable processors.

The exchange OKX said it will launch a new Blockchain OKBChain, which will be launched in Q1 and independent with current Okxchain. Okxchain is a POS chain and co-build by communities. OKBChain is developed and operated by OKX company.

JIDU, a leading robocar maker, is teaming up with Baidu to integrate ERNIE Bot and create the world's first LLM-powered natural communication experience for intelligent driving. This will take robocars to a new level of interactivity.

⚡️On June 1st, 2023, Hong Kong will officially make crypto purchase & sell, trading, fully legal for all of its citizens

Expect a huge influx of big money from the East. Asian currency based stablecoin coming out of HK will be a certainty as well.

Open call for 2023 internships (potentially followed by PhD) in Brain & AI team at Meta:
metacareers.com/jobs/849054792…

Strong experience in {deep learning, signal processing, neuroscience} (2 out of 3) required.

Application from under-represented groups encouraged.

Stability AI partners with Krikey AI to launch AI animation tools

Krikey's AI Text to Animation tool enables users to use a text phrase to generate avatar animations in minutes.

Users can then export a video file or fbx file to their social media account, 3D game engine or film editing software of choice.

Krikey's tool also allows users to customize their 3D Avatar and then generate an AI animation for that avatar.

The AI Animation tool launches today and is available at www.krikey.ai, more Avatar customization features will be launching this quarter.

Information on ChatGPT’s alignment, plans to improve it, giving users more control, and early thoughts on public input.

Google is offering an internal badge to their employees who help RLHF Bard.

And slowly, RLHF will eat the world.

Data flywheels are real - whether you’re labelling data for supervised learning or providing ground truth responses for large models, someone’s gotta do it.

Those who do it early and iterate fast seem to win.

GenAug is a new system by Meta AI researchers that uses text2image models to enable robots to transfer behaviors zero-shot from a simple demonstrated scene to unseen scenes of varying complexity — at no extra robot/human cost.

⚡️Co-founder of OpenAI - There’s a possibility that jailbreaks that work on advanced jailbreak-resistant models will also work on people

A good interview of Steven Pinker about ChatGPT, its potential uses and limitations.

He says, that LLMs make up stuff, make stupid mistakes, and don't have common sense, because they do not possess world models.

Hong Kong Plans to Let Retail Sector Trade Larger Crypto Tokens

Hong Kong outlined a plan to let retail investors trade digital tokens like Bitcoin and Ether, taking a major step toward its goal of becoming a crypto hub in a policy shift that contrasts with a crackdown in the US.

Individual investors would be allowed to trade larger coins on exchanges licensed by the Securities and Futures Commission, providing safeguards such as knowledge tests, risk profiles and reasonable limits on exposure are put in place, the regulator said in a consultation paper on Monday.

The agency didn’t specify which large-capitalization tokens will be allowed for retail investors. Instead, it said the coins should be included in at least two acceptable, investible indexes from independent providers, one of which should have experience in the traditional financial sector.

The consultation period will end on March 31, and the objective is to allow retail trading in the new licensing regime for crypto exchanges due on June 1.

Bitcoin and Ether, the two biggest digital assets by market value, are likely to be listed by Hong Kong platforms, an SFC spokesperson said in a briefing.

Pretraining Language Models with Human Preferences

Pretraining with human feedback results in much better preference satisfaction than standard LM pretraining followed by finetuning with feedback, i.e., learning and then unlearning undesirable behavior.

Web3 tech provider droppGroup signed a memorandum of understanding with energy giant Aramco to co-develop Web3 tech aimed at helping the energy giant's workers.

FAIR played a key role in making the AI R&D scene open. Others followed. At least for a while.

Now, OpenAI, DeepMind, & perhaps even Google are clearly publishing & open-sourcing considerably less.

What will be the consequences on the progress of AI science and technology?

TikTokShop’s transforming into an integrated shopping platform like Taobao’s Tmall, Shopee, encouraging user to actively search for products when shopping. Since entering the SEA mkt in 2022, its avg CMGR spiked to ~90% as Thailand & Philippines become its main mkts.

An interesting consequence of the wave of startups building on LLMs is that many YC startups are facing more technology risk than market risk.

The main question is not if people want it, but if it's possible to build it.

Futurism reports Neuralink is being investigated for contaminated hardware

The issue stems from improperly transported implants that may harbor infectious pathogens.